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Preface 


The  days  when  a  single  researdier,  working  alone  in  a  laboratory  could  make 
great  contributions  to  his  or  her  held  have  past.  Those  days  are  part  of  a  bygone  era 
or  perhaps  never  existed  at  all.  Great  accomplishment  today  is  the  result  of  team 
efforts,  each  individual  striving  to  make  a  contribution  by  fitting  a  small  piece  into  a 
larger  puzzle.  This  work  contains  the  small  piece  of  the  puzzle  I  have  worked  on.  Such 
work  would  not  be  meaningful  or  useful  without  strong  leadership  and  direction  from 
real  giants.  Dr.  Steven  K.  Rogers,  Dr.  Matthew  Kabrisky  and  Major  Phil  Ambum 
provided  that  leadership.  I  would  like  to  acknowledge  my  appreciation  for  their  help 
and  support.  Many  thanks  to  my  thesis  advisor,  Dr.  Steven  K.  Rogers,  for  his 
encouragement  and  enthusiasm.  I  would  also  like  to  thank  Dr.  Matthew  Kabrisky 
and  Dr.  Phil  Ambum  for  their  assistance  and  advice  throughout  the  research  effort. 

Sir  Isaac  Newton  once  said  “If  I  have  seen  farther  than  other  men  it  is  because 
I  have  stood  on  the  shoulders  of  giants”.  If  I  have  seen  at  all,  it  is  because  I  peered 
over  the  shoulders  of  giants. 
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Abstract 


An  environment  is  developed  for  the  study  of  dynamic  changes  in  patterns  of 
weight  and  node  values  for  artificial  neural  networks.  Graphic  representations  of 
neural  network  internal  states  are  displayed  using  a  high  resolution  video  terminal. 
Patterns  of  node  firings  and  diacges  in  weight  vectors  are  displayed  to  provide  in¬ 
sight  during  training.  Four  pattern  recognition  problems  are  applied  to  four  types 
of  artificial  neural  networks.  U.'iiig  simulated  data,  a  simple  disjoint  region  clasr  i- 
fication  problem  is  developed  and  examined  using  a  Kohonen  net  and  a  multilayer 
feedforward  back  propagation  (MFB)  network. 


A  MFB  neural  network  is  also  used  to  simulate  a  Fourier  filter.  Using  a  Koho¬ 
nen  net,  a  MFB,  a  counterpropagation  and  a  hybrid  network,  data  measured  from 
infrared  and  laser  radar  imagery  of  military  vehicles  is  analyzed.  The  accuracy  and 
training  times  for  a  MFB  net  and  a  Hybrid  net  are  compared  using  an  ambiguous 
decision  regiou  problem.  Each  classification  problem  is  examined  and  compared  to 
classical,  nearest  neighbor  pattern  recognition  techniques.  Using  dynamic  analysis, 
neural  network  pruning  is  used  to  determine  optimum  node  configurations.  A  hybrid 
neural  network  is  developed  using  Kohonen  training  rules  for  the  first  hidden  layer 
followed  by  one  or  two  hidden  layers  using  standard  back  propagation  n’.ies  for  train¬ 
ing.  Advantage  of  the  hybrid  network  is  shown  for  classification  problems  involving 
anomalies  characteristic  of  measured  data.  The  Hybrid  network  requires  less  train¬ 
ing  and  fewer  interconnections  than  MFB  when  classifications  involves  ambiguous 
decision  regions. 
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DYNAMIC  ANALYSIS  OF 
FEEDFORWARD  NEURAL  NETWORKS 
USING  SIMULATED  AND  MEASURED  DATA 


I.  Intrcduction 

Autonomous  military  target  detection  and  classification  from  electronic  im¬ 
agery  is  a  topic  of  great  importance  to  the  Department  of  Defense.  The  solution  to 
the  problem  may  lie  in  one  of  several  implementations  of  artificial  neural  networks. 
Several  topologies  for  neural  networks  have  been  proposed,  each  of  which  provide 
a  solution  for  a  narrow  class  of  pattern  recognition  problems.  Some  researchers 
(Huang,1987)  feel  that  combinations  of  more  than  one  type  of  neural  network  may 
result  in  a  more  dynamic  and  robust  system. 

Identification  and  classification  of  targets  from  electronic  imagery  is  a  difficult 
problem  due  to  the  vast  amounts  of  data  involved.  A  single  image  can  contain  mil¬ 
lions  of  bits  of  information,  all  of  which  need  to  be  processed.  Processing  images 
for  pattern  recognition  is  a  threefold  problem.  First,  the  targets  must  be  separated 
from  the  background  or  segmented.  Second,  the  data  must  be  reduced  to  a  man¬ 
ageable  size,  commonly  called  vector  quantization.  This  reduction  in  data  can  be 
accomplished  by  selecting  specific  features  of  a  pattern  and  using  only  these  features 
for  classification.  Good  pattern  recognition  requires  good  f<.  atures.  The  final  task  i.s 
classification  of  the  vectors. 

Determining  which  I'eatures  of  an  image  form  the  best  description  of  an  object 
is,  in  itself,  a  difficult  problem.  This  study  will  examine  several  sets  of  data  collected 
from  a  variety  of  sensors,  including  laser  radar  and  passive  infrared  imagery.  Baseline 


clAssificAtios  analysis  will  be  made  usint,  simple  feedforward  neural  network*  then 
extended  to  new  forma  ot  combinations  of  the  feedf'>rward  networks. 

One  of  the  difficulties  encountered  when  testing  neural  networks  is  the  lack  of 
good  test  data.  Training  a  pattern  recognition  system  requires  thousands  of  teach¬ 
ing  cycles.  Although,  one  would  piefer  very  large  training  sets  using  real  imagery, 
in  practice  infcrtunatdy,  training  sets  rarely  ex  'eed  more  than  a  fe\-.’  hundred  im¬ 
ages  due  to  the  difficulty  in  segmentation  and  vector  quantization.  When  exploring 
or  testing  a  particular  neural  net  algorithm,  the  data  problem  can  be  avoided  by 
computer  generation  of  the  input  vectors,  based  on  the  problem  description. 

Success  of  a  particular  classification  problem  depends  on  a  number  of  factors. 
First,  consider  the  validity  of  the  segmentation  of  the  data.  Has  the  actual  target 
been  separated  from  the  background  data  and  noise  ?  Next,  is  the  feature  extraction 
legitimate.  Do  the  features  selected  for  the  input  vector  represent  a  good  description 
of  the  target  ?  Once  the  target  has  been  extracted  from  the  background,  is  the  vector 
quantized  description  unique  enough  to  allow  classification  ?  Finally,  is  the  neural 
network  topology  sufficient  for  the  size  of  the  decision  region  and  can  it  accurately 
classify  input  patterns  ?  Special  tools  may  be  needed  to  answer  these  questions. 

An  environment  to  examine  internal  operation  of  the  neural  networks  as  they 
train  could  help  determine  thr  efficiency,  and  accuracy  of  different  topologies.  Eval¬ 
uation  of  the  internal  constants  and  variables  as  the  network  trains  may  offer  insight 
into  which  values  may  be  best  suited  for  a  particular  set  of  circumstances.  Although 
the  primary  area  of  investigation  considered  here  is  network  topologies,  the  soft¬ 
ware  package  generated  as  a  result  of  this  effort  will  be  a  generalized  research  tool 
to  study  segmentation  or  vector  quantization  algorithms  as  they  apply  to  neural 
network  classifi<.  ion  problems. 


1.1  Baekgro%nd 

The  Ability  of  machinea  to  interptet  visual  images  remains  an  unsolved  prob¬ 
lem.  Military  planners  have  long  been  interested  in  developing  means  to  automati¬ 
cally  detect  and  classify  military  targets  using  conventional  sensors.  Although  these 
aensofi,  tdevision,  infrared  scanners  or  multifunction  laser  radars,  provide  enough 
information  for  a  human  operator  to  find  a  target,  the  exte*^r?on  to  automatic  detec¬ 
tion  and  classification  is  stiU  impractical  using  current  computer  architectures.  The 
oomputatioaal  effort  to  classify  or  detect  tactical  targets  firom  mission  sensor  data  is 
too  demanding  to  be  performed  in  real-time. 

These  problems  may  be  resolved  by  a  computer  technology  called  artificial 
neural  networks.  Artificial  neural  networks  ate  computer  structures  or  architectures 
that  attempt  to  mimic  some  of  the  known  characteristics  of  biological  brains.  Neural 
networks  may  provide  rdatively  fast,  i^projamate  solutions  to  problems,  as  opposed 
to  the  slow,  exhaustive  (but  exact)  solutions  provided  by  conventional  computer 
architectures.  Artificial  neural  networks  may  be  arranged  in  a  variety  of  intercon¬ 
nections  of  data  input  and  outputs  along  with  intermediate  levels.  These  different 
arrangements  characterize  the  topology  of  the  network.  Of  the  myriad  of  topologies 
of  neural  networks  studied,  the  most  directly  applicable  to  the  target  identification 
problem,  are  the  multilayer  feedforward  networks  (MFB)  using  backward  error  prop¬ 
agation  (sometimes  called  multilayer  perceptrons)  and  the  Kohonen  maps.  These 
two  classes  of  neural  networks  have  an  advantage  over  many  other  networks,  as  both 
accept  continuous  data  as  bput  (Lipproan,1987).  Many  other  network  configura¬ 
tions  accept  only  binary  data.  Although  binary  data  processing  is  sufficient  for  some 
classes  of  problems  such  as  text  recognition,  to  pick  an  object  out  of  an  image,  the 
system  must  be  able  to  process  continuous  valued  inputs.  The  analog  data  pro¬ 
cessed  by  the  network  may  be  in  the  form  of  correlation  peaks,  statistical  moments, 
or  some  other  vector  quantization  of  distinguishing  characteristics  calculated  from 
the  original  image. 


While  both  the  multilayer  perceptron  and  the  Kohonen  maps  are  suited  to 
the  pattern  recognition  problem  there  are  drawbacks.  The  MFB  model  requires 
hundreds  of  thousands  of  training  cycles.  Complete  training  may  require  sc/eral 
days  on  a  high  speed  computer.  Also,  while  the  number  of  nodes  at  the  input  and 
output  is  defined  by  the  problem,  optimization  of  the  inner  node  parameters  has  not 
been  fully  addressed.  The  number  of  input  nodes  is  fixed  by  the  size  of  the  input 
vector  and  the  number  at  the  output  relates  to  the  number  of  classes.  Currently,  the 
correct  number  of  hidden  nodes  i.-:  determined  by  experimentation.  Some  feel  that 
the  number  of  hidden  nodes  in  the  ^t  layer  should  be  three  times  the  number  in 
the  input  layer.  Others  feel  the  configuration  of  the  inner  layers  should  be  related 
to  the  input  data  (Baum,1986). 

Analysis  of  data  using  a  Kohonen  m^  only  partially  solves  the  problem.  The 
output  of  a  Kohonen  map  is  a  two  dimensional  mapping  of  a  multi-dimensional 
decision  space.  By  mi^iping  one  class  into  one  area  of  the  map  and  another  class 
into  another  r^ion  of  the  map,  the  complexity  of  the  decision  regions  are  reduced. 
Unfortunately,  for  a  complete  solution  the  map  must  be  interpreted.  Currently,  that 
is  done  by  examination.  The  output  of  a  Kohonen  map  is  simply  another  type 
of  pattern  recognition  problem.  Huang  bdieves  that  a  Kohonen  map  may  be  able 
to  function  as  a  preprocessor  fur  some  other  class  of  neural  network  (Huang,1987). 
The  combination  o(  the  Kohonen  map  and  the  multilayer  perceptron  may  offer  a 
solution  to  the  weaknesses  of  both.  Using  a  Kohonen  map  to  organize  the  data 
into  a  two  dimensional  grid,  then  feeding  the  output  of  the  grid  nodes  to  the  input 
layer  of  a  multilayer  perceptron  provides  a  mean  of  interpreting  the  Kohonen  map. 
The  Kohonen  map  may  also  reduce  the  complexity  of  the  decision  space  for  the 
perceptron.  Training  time  should  be  reduced  since  the  time  to  train  the  weight 
values  for  a  perceptron  is  related  to  the  complexity  of  the  decision  regions. 
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J.S  Problem 


The  largest  part  of  the  effort  will  be  devoted  to  developing  a  method  to  display 
the  intonal  values  of  the  network  using  a  graphic  representation  that  allows  insight 
into  it’s  operation. 

By  analysis  of  the  dynamic  nature  of  the  networit,  it  is  hypothesized  that 
methods  for  optimizing  the  topology  of  the  network  may  be  devised,  and  a  bet¬ 
ter  understanding  of  the  artificial  neural  network  paradigms  would  be  gained.  The 
problem  of  general  interest  is:  can  two  t]rpes  of  neural  network  topcdogies  be  com¬ 
bined  to  get  a  synergistic  effect  greater  than  the  sum  of  the  two  separately?  The 
MFB  provides  a  solution  to  the  disjoint  decision  region  problem  while  the  Kohonen 
.-nap  provides  a  method  to  organize  unclassified  data.  This  thesis  effort  win  examine 
several  methods  of  combining  networks  to  get  improved  pofermance  and  reduced 
training  time. 

i.S  Afproaeh 

The  final  product  of  the  study  is  a  graphics  intensive  environment  for  dynamic 
analysis  of  artificial  neural  networks.  This  software  paduige  will  allow  the  user  to 
explore  other  neural  netwwk  problems  using  common  problem  definition  format.  By 
creating  a  file  listing  of  a  number  of  exemplars  patterns  (vector  quantizations),  along 
with  a  specified  classification,  any  general  classification  problem  can  be  feed  into  the 
package  for  analysis. 

The  package  will  consist  of  four  t]rpes  of  neural  networks  accessed  from  a 
common  menu.  The  networks  are  a  Kohonen  map,  a  multilayer  perceptron,  a  counter 
propagation  net  and  a  hybrid  net.  In  addition,  an  error  surface  demonstration  will 
be  included  in  the  menu  for  tutorial  purposes. 

Validation  and  testing  of  the  package  is  accomplished  by  using  the  software  to 
study  four  neural  net  problems.  The  first  problem  considered  is  a  decision  region 
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probleDOt  using  both  simple  and  disjoint  decision  regions.  The  second  problem  will 
consider  neural  networks  as  a  Fourier  filter.  The  last  two  problems  will  use  measured 
image  data  from  the  Ruck  (Ruck,19fi7)  and  Roggemann  (Roggemann4983)  data  seta. 

Using  the  environment  tools,  efforts  will  be  made  to  optimize  the  number  of 
nodes  to  the  type  of  data  being  tested.  Each  network  will  be  tested  for  the  number 
of  training  cycles  to  convergence,  and  accuracy.  Test  sets  will  be  analyzed  using  each 
of  the  conventional  nets,  as  well  as  classical  neatest  neighbor  classifiers. 

The  devdopment  of  a  hybrid  neural  network  will  be  the  final  itep  in  a  process 
to  examine  several  types  of  neural  networks.  The  process  will  include  finding  ways 
to  display  information  about  the  dynamic  processes  inside  the  networks. 

Chapter  II  provides  background  information  concerning  artificial  neural  net¬ 
work.  Neural  networks  are  discussed  in  context  of  the  type  of  problems  each  topology 
is  intended  to  solve. 

Chapter  III  discusses  the  software  engineering  aspect  of  the  neural  network 
environment.  The  organization  of  the  software  is  explained  in  terms  of  the  various 
functions  sod  operation  of  the  major  modules. 

Chapter  IV  provides  the  analysis  and  testing  of  the  software  environment. 
Several  common  classification  problems  are  analyzed  using  the  system  as  well  as  two 
specific  data  sets.  Each  problem  or  data  set  is  analyzed  using  one  of  several  common 
neural  network  topologies. 

Chapter  V  is  an  overview  of  the  results  of  the  experiments  including  general 
observations  ^plicable  to  many  t^'pes  of  neural  network  problems. 
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IJ.  Literatun  Review 


Rul-time  twget  recognition  f<»  intdligent  weapons  systems  is  too  compu¬ 
tationally  intensive  to  be  practical  using  current  computer  technology.  Picking  a 
legitimate  military  target  from  video  sensor  information  requires  aj  many  calcula¬ 
tions  that  identification  and  classification  may  requires  hours  while  only  seconds 
are  available  The  brain,  a  biological  computing  engine,  can  solve  these  type  prob¬ 
lems  quickly.  A  novd  computer  architecture  based  on  the  way  the  brain  is  thought 
to  function,  may  provide  a  solution  to  classification  problems.  This  architecture, 
called  artificial  neural  networks,  may  provide  an  alternate  approach  to  conventional 
computers  and  artificial  intelligence  paradigms  for  target  recognition  tasks. 

This  section  will  discuss  the  origins  (rf  the  artificial  neural  networks,  and  the  re¬ 
lation  between  the  bidogical  inspiration  and  the  computer  implementations.  Start¬ 
ing  with  the  basic  building  blocks  of  neural  computers,  the  node,  the  difference 
between  natural  and  artifidal  intelligence  will  be  discussed  in  terms  of  abstract  rea¬ 
soning  verses  brute  force  calculations.  Several  topologies  of  node  interconnections 
will  be  discussed  to  demonstrate  the  diversity  of  programming  or  training  tech¬ 
niques.  Finally,  this  chapter  will  discuss  the  characteristic  of  problems,  in  terms  of 
disjoint  and  ambiguous  decision  regions,  which  affect  selection  of  network  topology 
and  training  rules. 


2.1  Historical  Perspective 

Greek  philosophers,  including  Plato  and  Aristotle,  offered  theoretical  expla¬ 
nations  of  the  brain  and  thinking  process  a  few  thousand  years  ago.  Heron  the 
Alexandrian  built  a  hydraulic  automata  around  100  B.C.  Abstract,  conceptual  in¬ 
formation  processing  operations  have  been  performed  by  mechanical  devices  for  a 
few  centuries,  for  example  the  slide  rule. 
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The  first  theorists  to  conceive  of  a  computer  based  psychological  models  and 
neurophysiological  research  were  W.S.  McCulloch  and  W.A.  Pitts  in  the  early  1940’s 
(Kohonen,1987).  Although  early  research  efforts  parallek-d  the  development  of  the 
modem  computer  (the  Von  Neumann  Machine ),  the  concept  of  a  cybernetic  machine 
has  been  popular  since  ancient  times.  These  types  of  computers  were  first  called 
connectionist  machines  and  later  artificial  neural  networks. 

Artificial  neural  networks  have  been  studied  for  “everal  decades.  At  first  they 
were  studied  in  connection  with  psycholopcal  theories  and  neurophysiological  re¬ 
search.  By  1960,  many  implementations  of  "neural  computers”  had  been  developed. 
Today,  neural  computers  may  offer  a  solution  to  the  growing  need  for  a  machine 
that  will  perform  not  only  calculations,  but  actually  make  judgments.  A  funda¬ 
mental  limitation  of  conventional  computer  architectures  is  the  inability  for  abstract 
reasoning.  Based  on  the  biological  model,  neural  network  computers  attempts  to 
overcome  that  weakness. 

8.S  Natural  vs  Artificial  Intelligence 

Although  it  may  be  impossible  to  determine  exactly  how  the  brain  works, 
observation  of  its  behavior  may  be  enough  to  determine  why  humans  are  good  at 
solving  types  of  problems  that  cause  computers  great  difficulty.  Identifying  a  face  in 
a  crowd,  or  navigating  through  a  room  without  bumping  into  the  furniture,  requires 
an  effort  beyond  the  capability  of  modem  computers.  Yet,  even  children  leam  to 
identify  their  parents  after  only  a  few  months  and  the  smallest  insect  is  capable  of 
solving  navigation  problems  beyond  the  capabilities  of  advanced  robotic  systems. 

Bruce  D.  Shriver  noted: 

. . .  digital  computers  are  extremely  good  at  executing  sequences  of  in¬ 
structions  that  have  been  precisely  formulated  for  them  with  stored  pro¬ 
grams  representing  the  processing  steps  that  need  to  be  done.  The  human 
brain,  on  the  other  hand,  performs  well  at  such  tasks  as  vision,  speech, 
information  retrieval,  and  complex  spatial  and  temporal  pattern  recog- 
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nition  in  the  presence  of  noisy  &nd  distorted  data- tasks  that  are  very 
difficult  for  sequential  digital  computers  to  do  at  all  (Caudill,  1987:48). 

Artificial  intelligence  is  a  sequential  process  which  involves  collecting  all  avail¬ 
able  data  and  systematically  processing  each  piece  until  the  solution  is  established 
by  completing  an  algorithm.  Natural  intelligence  differs  from  artificial  intelligence 
by  using  a  system  to  extract  rdevant  information  from  the  available  data,  then  ex¬ 
trapolate  an  approximate  solution.  In  terms  of  the  target  classification  problem, 
the  process  would  organize  the  data  (self-organization),  extract  relevant  information 
(feature  extraction),  and  compute  a  solution  (classification). 

The  basic  building  blocks  used  by  the  brain  to  perform  these  calculations  are 
called  neurons. 

2.S  The  Neuron 

.  Neurons  are  the  brain’s  electrochemical  processing  elements  which  allow  indi¬ 
viduals  to  “store,  represent,  retrieve,  and  ounipulate  data  such  as  images,  smdls, 
sensations  and  thoughts”  (Caudill,  1987:48).  Neurons  are  slow  by  digital  electronic 
standards.  Response  times  ate  measured  in  hundreds  of  milliseconds  as  opposed  to 
the  nanosecond  response  time  of  today’s  integrated  circuits.  Still,  the  massive  num¬ 
ber  of  neurons  may  make  up  for  the  lack  of  speed.  The  brain  is  estimated  to  use 
between  lO'”  and  10*'  neurons  (Kohonen, 1987:227)  in  an  intricate,  interconnected 
structure.  Many  neurons  are  connected  to  thousands  of  other  neurons.  Each  neuron 
reaching  out  to  other  neurons  by  means  of  a  single  output,  the  axon,  a  fiber-like  struc¬ 
ture  that  attaches  to  other  neurons  by  a  synaptic  terminal  (Kohonen,1987,:210-240). 
This  connection  between  the  neurons  performs  a  small,  imprecise  multiplication  of 
the  signal  transmitted  down  the  axon  to  the  synaptic  terminal.  These  signals  are 
accumulated  by  the  receiving  n>iuron.  When  the  sum  of  these  signals  reach  a  suffi¬ 
cient  level,  the  neuron  fires  a  weak  electrical  signal  down  its  own  axon  to  the  next 
neuron  in  the  chain. 


Th'.  process  causing  the  neuron  to  fire  is  the  most  powerful  aspect  of  neural 
'‘omputing.  Each  of  the  axons  provide  an  adapt^le  weight  which  can  inhibit/exdte 
the  signal  between  neurons.  The  axons  are  similar  to  the  memory  unit  in  a  digital 
computer  with  one  exception;  they  are  able  to  adapt  and  ch  uge  with  time.  The 
mechanism  which  modifies  these  weights  enables  learning.  By  duplicating  this  pro¬ 
cess  artificially,  we  may  be  able  to  build  computers  that  learn  by  example,  rather  than 
loading  a  stored  program.  The  Japanese,  in  their  announcement  of  the  fifth  genera¬ 
tion  computers,  “coined  the  term  natural  vs  artificial  intelligence”  (Caudill,1987:46). 

What  kind  of  process  or  how  the  process  allows  people  to  think,  walk,  or 
recognize  objects,  is  not  understood.  That  this  process  contains  the  solution  to 
the  pa.tem  recognition  problem  is  demonstrated  by  our  own  ability.  Whether  a 
computer  can  reproduce  this  ability  is  the  subject  of  many  research  efforts.  Using 
a  neural  computer  architecture,  the  computer  may  be  able  to  discover,  on  its  own, 
the  underlying  structures  in  a  given  set  of  bput  data  which  makes  recogmtion  of 
specific  patterns  possible  (Kohonen,  1981:214-215). 

The  biological  neuron  is  the  basic  computation  unit  of  the  brain  and  provides 
the  inspiration  for  the  neural  network  node.  The  node  is  the  basic  computational 
element  of  artificial  neural  networks. 

2.4  The  Neund  Network  Node 

Advances  in  many  fields  have  made  possible  faster  and  more  powerful  com¬ 
puters  than  ever  before.  Unfortunatdy,  even  the  most  sophisticated  computers  are 
tied  to  a  structure  which  allows  them  to  perform  only  a  single  digital  operation  at 
a  time.  These  register  functions,  multiplication  of  two  numbers,  storing  results  or 
gett>  %  more  data,  must  be  performed  sequentially,  one  register  at  a  time.  Although 
the  speed  of  these  simple  computations  is  limited  only  by  the  speed  of  the  material 
and  design  technology,  there  is  only  a  single  path  for  the  data  flow. 

This  limitation  is  known  as  the  “Von  Neumann  Bottleneck"  (Hamacher,1986). 
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Figure  1.  A  Neural  Network  Node 

Because  each  calculation  must  pass  through  a  single  point,  the  central  processor, 
data  flow  is  restricted,  Uke  a  bottleneck.  This  obstacle  may  soon  be  overcome  by 
an  architecture  which  uses  a  massively  parallel  approadi  to  problem  solving.  These 
structures,  called  Neural  Networks  or  Connectionist  Madiines,  are  based  at  least 
partially,  on  the  way  the  human  brain  is  thought  to  function.  The  simplest  processing 
element  of  a  neural  network  is  the  node.  Schematically,  a  node  is  composed  of  a 
multitude  of  inputs  and  one  output. 

In  early  neural  networks,  the  output  node  value  was  either  zero  or  one,  based 
on  thresholding  the  dot  product  of  the  input  vector  and  the  weight  vector.  A  neural 
computer  is  composed  of  from  several  to  a  few  thousand  nodes. 

Dr.  Robert  Hecht-Nielsen  describes  the  first  commercial  neurocomputer  as 

“a  computing  system  made  up  of  a  number  of  simple,  highly  intercon¬ 
nected  processing  elements,  which  processes  information  by  its  dynrunic 
state  response  to  external  inputs”  (Caudill,1987). 
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By  connecting  the  output  of  some  nodes  to  the  inputs  of  other  nodes,  Inrge 
arrays  or  networks  are  formed  which  seem  to  be  capable  of  computing  abstract 
decisions.  Each  processing  element  computes,  in  parallel  with  all  other  nodes,  a 
simple  true/ false  decision  based  on  the  weighted  sum  of  the  inputs.  These  basic 
decisions  form  a  basis  for  broader  abstract  reasoning. 

Analysis  of  the  differing  methods  to  arrange  the  interconnection  between  nodes 
together  with  the  rules  to  establish  the  interconnection  weights  makeup  a  large  part 
of  the  study  of  artificial  neural  networks. 

2.5  Artificial  Neural  Networks 

Study  of  neural  network  models  is  usually  conducted  on  standard  digital  com¬ 
puters.  Sev  jral  of  Kohonen’s  models  were  first  written  in  Turbo  Pascal  for  the  IBM 
personal  computer  (Kohonen,  1987:14).  These  modeb  take  the  form  of  a  computer 
program.  The  various  parts  of  the  neuron  are  modeled  as  daM  structures  in  digital 
memory.  Although  many  of  the  advantages  of  using  neural  nets  are  lost  by  using  a 
“Von  Neumann”  machine,  the  research  process  may  devebp  algorithms  latr'  suitable 
for  dedicated  hardware. 

Devdoping  a  neural  net  is  a  three  step  process.  First,  the  data  structures 
are  organised  in  memory,  either  as  arrays  or  link  lists.  There  are  two  t3rpes  of 
data  structures,  the  nodes  and  weights.  The  nodes  are  simple  accumulators  which 
occupy  the  number  of  memory  locations  or  bytes  required  for  one  floating  point 
number.  Nodes  be  considered  the  “neurons”  of  a  neural  network.  The  other 
data  structure,  the  weights,  are  analogous  to  the  axons.  They  occupy  an  array  of 
floating  point  memory  locations,  usually  a  two  dimensional  matrix  of  floating  point 
numbers.  The  second  step  is  to  determine  the  adaptive  weight  values  using  a  training 
algorithm.  These  can  be  either  supervised,  that  is,  a  priori  knowledge  of  the  input 
pattern,  or  unsupervised,  using  unclassified  random  inputs.  Finally,  using  a  black 
box  approach,  data  is  presented  to  the  inputs,  propagated  through  the  system,  then 


r«ul  the  outputs  (Konoaen,I987:13-l7). 

Like  the  neuron  in  biological  systems,  the  node  is  the  principle  unit  of  artificial 
systems.  A  node  value  is  formed  by  mathematically  manipulati  the  input  value  and 
the  connection  weights.  Exactly  how,  is  dependent  on  a  particular  neural  network 
topology.  Every  element  of  the  input  vector  is  connected  to  every  node.  Many 
arrangements  or  taxonomies  of  nodes  and  wdghts  have  been  developed  by  Hopfield 
and  others.  See  Lippman’s  overview  (Lippmao,  1987:7-13). 

Each  type  of  neural  network  is  applicable  to  a  specific  class  of  recognition  prob¬ 
lems.  Only  two  of  these  networks  relate  to  the  problem  under  investigation,  Kohonen 
maps  and  the  multilayer  feedforward  backward  error  propagation  nets  (MFB),  some¬ 
times  called  the  Multilayer  Perceptron  net.  These  two  neural  network  topologies  are 
suitable  to  the  combination  network  problem  because  they  allow  continuous  data 
to  be  used  as  inputs.  Most  other  types  of  networks  accept  only  binary  data  (Lipp- 
mann,  1987:7-13).  The  Kohonen  map  was  selected  for  its  ability  to  organize  data 
into  rational  decision  r^ions,  while  the  MFB  was  selected  for  its  ability  to  classify 
data  contained  in  complex  decision  repons.  Classification  efforts  could  be  reduced 
if  relevant  features  which  differentiate  the  exemplars,  could  be  extracted  from  the 
entire  data  set  first. 

This  is  the  principle  of  self-organization.  Self-organization  is  a  form  of  unsu¬ 
pervised  learning.  In  unsupervised  learning,  the  classification  of  a  particular  training 
exemplar  is  not  used  in  training  process.  Comparison  of  the  actual  output  with  a  de¬ 
sired  output,  essential  to  backward  error  propagation,  is  not  made.  Self-organization 
on  compares  e«ch  exemplar  with  all  the  other  exemplars.  Unsupervised  learning  pro¬ 
vides  an  organization  of  the  data  which  reduces  the  complexity  of  the  classification 
problem. 


2.6  Unsuptrviatd  Leaminij 

Kohonen  fdt  that  the  pattern  recognition  problem  could  be  simplified  if  the 
algorithm  could  somehow  automatically  extract  relevant  features  from  the  input  data 
(Kohonen,  1981).  Kohonen ’s  work  differed  from  earlier  work,  because  he  was  looking 
for  an  organization  of  neuron-like  units  which  could  automatically  detect  common 
characteristics  of  the  input  data  without  regard  to  a  specific  classification.  As  he  was 
trying  to  do  speech  recognition,  his  data  set  consisted  of  many  unlabeled  examples. 
Kohonen  was  using  input  vectors  made  by  spectral  decomposition  of  speech  to  train 
his  mapping  networks.  These  examples  were  presented  to  the  recognizer  in  hopes 
that  the  mapping  algorithm  could  sort  out  the  various  vowel  and  consonant  sounds. 
The  Kohonen  map  tries  to  construct  a  map  or  “hierarchical  clustering”  of  patterns 
with  the  same  characteristics.  He  called  this  feature  extraction  the  “first  step  to  all 
perception”  (Kohonen  1981:2).  To  do  this,  he  developed  an  unsupervised  learning 
scheme  which  pushes  one  class  of  data  to  one  part  of  the  map  while  another  class 
would  appear  on  another  part  of  the  map.  These  “feature  maps”  are  a  spatial 
clustering  of  input  samples  with  similar  characteristics. 

2.7  Kohonen  Feature  Maps 

A  Kohonen  map  has  a  unique  structure.  A  layer  of  at  least  two  input  nodes  is 
required.  Weights  connect  the  inputs  to  a  layer  of  output  nodes.  Every  input  node 
is  connected  to  every  output  node.  This  output  layer  is  constructed  in  the  form  of  a 
grid.  Fifteen  by  fifteen  nodes  is  a  common  arrangement  (Barmore,1988). 

The  map  is  trained  by  using  distance  measurements  between  the  input  data 
and  the  waghts.  The  difference  is  fed  back  to  the  system  to  reduce  the  distance  for 
inputs  with  similar  chrracteristics.  When  data  is  presented  to  the  input  nodes,  the 
Euclidian  distance  between  an  input  and  a  output  is  calculated.  This  distance  is 
calailated  by  considering  each  output  node  in  turn.  First,  calculate  the  sum  of  the 
square  of  the  differences  between  each  input  node  and  the  weight.  After  considering 


each  output  node,  one  distance  will  be  lower  than  any  of  the  others.  This  node  is 
considered  the  “winner.”  The  weights  connected  to  this  node  are  adjusted  slightly 
so  the  distance  is  a  little  smaller.  Also,  each  node  in  a  small  neighborhood  around 
the  “winner”  is  adjusted  so  that  any  similar  data  will  appear  in  this  region.  As  the 
training  continues,  the  neighborhood  around  the  winner  is  reduced  until  only  the 
single  node  is  updated.  By  the  end  of  the  training  process,  under  non-pathological 
conditions,  similar  exemplars  will  cause  a  node  to  fire  in  small  region  of  the  map. 
Classification  is  a  matter  of  determining  which  region  of  the  map  is  activated  when 
data  is  presented  to  the  inputs. 

Notice  that  the  map  is  never  told  what  pattern  it  was  training  on.  The  net 
simply  organized  dissimilar  data  into  different  regions  of  the  map.  Kohonen  classifi* 
cation  is  based  on  visual  inspection  of  the  feature  map.  To  automate  the  classification 
process,  a  new  type  of  learning  process  is  required. 

2.8  Supervised  Learning 

Another  class  of  training  algorithms  uses  a  supervised  training  technique.  With 
this  method,  a  pattern  is  presented  to  the  input  of  a  neural  net,  then  the  difference 
between  the  actual  output  and  the  expected  output  is  calculated.  This  measure  of 
error  can  be  used  to  adjust  the  weights.  This  is  called  “error  correction”  or  “back 
propagation”.  Back  propagation  is  the  principle  behind  the  single  and  multilayer 
feedforward  (MFB)  model  neural  network. 

2.9  The  Single  Layer  Perceptron 

While  the  Kohonen  model  uses  unsupervised  learning  to  organize  data,  the 
specific  classification  of  an  exemplar  input  is  important  to  the  training  of  the  MFB 
model.  The  back  propagation  scheme  trains  the  neural  network  by  example.  This 
algorithm  begins  with  the  connection  weights  set  at  small  random  values.  Data  are 
presented  to  the  input  and  the  error  at  the  output  is  measured.  The  gradient  of  the 


error  is  calculated,  and  the  weights  are  adjusted  a  small  amount  to  reduce  the  total 
error.  New  data  are  presented  over  and  over  again,  correcting  the  connection  weights 
a  little  each  time.  Eventually  the  weights,  under  sufficient  conditions,  will  converge 
to  a  solution.  When  data  are  presented  to  the  input,  a  classification  value  will  appear 
at  the  output.  By  arranging  cne  set  of  input  nodes  to  feed  many  output  nodes,  many 
classes  of  patterns  can  be  identified  by  detecting  either  a  zero  or  a  one  at  the  output. 
Unfortunately,  the  work  of  Sun  (Sun,  1986)  and  others  (Lippman,  1987)  have  shown 
that  the  Single  Layer  Perceptron  can  only  make  correct  classifications  in  very  simple 
decision  regions.  A  number  of  complications  can  affect  the  performance  of  a  single 
layer  perceptron.  Different  classes  of  data  being  meshed  in  a  small  decision  region 
are  one  example.  The  next  section  discusses  a  class  of  problems  which  cannot  be 
solved  by  a  single  layer  perceptron:  the  disjoint  region  problem,  sometimes  called  a 
disjunctive  learning  region. 

2.10  The  Disjunctive  Learning  Region  Problem 

A  common  example  of  a  disjoint  region  problem  is  shown  in  figure  2.10.  This 
example  u  sometimes  called  the  exclusive  —  or  problem. 

Assume  a  specific  pattern  to  be  in  a  bounded  region  in  the  x-y  plane.  The 
input  vector  can  be  generated  by  selecting  one  of  the  regions,  and  adding  random 
noise.  The  output  vector,  or  desired  classification  of  the  input  vector  is  determined 
by  noting  the  location  of  the  x-y  pair  in  the  pleme.  The  example  can  be  extended  to 
include  any  number  of  disjoint  regions  in  either  two-space  as  described  above,  or  up 
to  n-space  with  n  elements  of  the  input  vector. 

Additionally  the  output  vector  can  describe  any  number  of  classes.  The  sim¬ 
plest  example  could  be  to  relate  one  region  to  one  class.  The  disjoint  region  problem 
concerns  the  case  where  two  regions  in  the  decision  space  make  up  a  single  class. 
Large,  even  infinite  data  sets  can  be  used  when  the  particular  problem  being  inves¬ 
tigated  allows  computer  generated  input  vectors  such  as  decision  regions  in  space. 
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Double  disjoint  decision  region 


Note:  No  single  pl&ne  can  divide  the  region  by  class 


Figure  2.  The  Disjoint  Region  Problem 
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The  disjunctive  learning  reipon  problem  stems  from  a  pattern  recognition  sit¬ 
uation  in  which  the  decision  space  may  be  split  into  several  regions.  Even  though 
two  or  more  regions  in  the  decision  space  may  be  widely  separated,  the  regions  may 
represent  the  same  class  of  patterns.  When  training  a  net  under  these  conditions, 
the  convergence  time  is  greatly  increased  and  may  not  be  possible  at  all.  Valiant 
notes: 

“Simple  rules  of  thumb  may  be  hard  to  learn.  There  is  evidence  that  cer¬ 
tain  significant . . .  classes  [may  be]  rendered  computationally  intractable” 
(Valiant,1983) 

He  was  referring  to  the  fact  that  generally  "rule  of  thumb”  type  decisions  fall  into 
this  class  and  contain  disjunctive  learning  regions.  Althouga,  La  some  cases  the 
network  can  learn  the  disjunction  in  the  decision  space,  the  training  time  may  be 
unacceptable. 

The  pierformance  of  the  perception  can  be  improved  by  adding  additional  lay¬ 
ers.  Single  layer  perceptrons  cannot  solve  disjunctive  learning  region  problems.  By 
adding  an  additional  hidden  layer  the  perception  model  can  be  use  to  solve  disjunc¬ 
tive  learning  regions.  The  next  section  explains  the  multilayer  perception. 

2.11  Midtilayer  Perceptron 

Adding  layers  to  the  perception  model  can  allow  for  an  increase  in  the  complex¬ 
ity  of  the  decision  regions.  The  input  to  a  perception  can  be  considered  as  a  point 
in  a  decision  space  with  one  input  node  for  each  dimension  of  the  decision  space. 
The  weight  vector  defines  a  place  (or  hyper-plane  for  n-dimensional  space)  dividing 
the  decision  space  into  two  regions  (Lippmann,1987:16).  Inputs  whose  hyperspace 
representation  appear  on  one  side  of  the  plane  are  considered  to  be  a  member  of  the 
class  or  in-class  and  inputs  on  the  other  side  of  the  plane  are  considered  out-of-class. 
Additional  layers  in  the  perception  model  allow  for  more  complex  decision  regions  by 
partitioning  off  the  space  into  intersections  of  several  in-class  regions.  The  network 


topology  is  changed  by  adding  one  or  two  hidden  layers  of  nodes  between  the  input 
and  output  layer.  Minsky  and  Pappert  showed  that  a  single  layer  perception  could 
not  be  used  to  solve  the  exclusive-or  problem  (Minsky, 1961).  Huang  and  Lippmann 
showed  that  two  and  three  layer  nets  can  form  arbitrary  decision  regions  as  well  as 
solve  disjoint  decision  region  problems  (Haung,1987:l*2). 

A  problem  may  arise  when  the  inpnt  data  is  not  suffidently  separable.  Rosen¬ 
blatt  demonstrated  (Lippmaim,  1987:14)  that  if  the  data  was  separable,  and  a  boimd- 
ary  could  be  placed  between  the  two  decision  regions,  the  training  algorithm  would 
converge  to  the  correct  solution.  However,  if  the  data  is  not  sufficiently  separable, 
then  the  convergence  procedure  might  oscillate,  moving  the  decision  boundary  be¬ 
tween  overlaying  data  points.  A  modification  to  the  training  algorithm,  using  a  least- 
means  squares  solution  was  suggested  by  Widrow  and  Hoff  (Lippmaim,  1987:14). 
Unfortunately,  the  algorithm  is  not  as  efficient  for  distinctly  separable  data.  This 
type  of  problem  is  caused  by  ambiguous  decision  regions.  When  two  d asses  of  data 
are  very  close  to  each  other  in  decision  space,  perceptnm  models  do  not  converge 
well  as  will  be  shown  in  Chapter  four. 

Another  solution,  recommended  by  Huang,  suggests  using  a  hybrid  combina¬ 
tion  of  a  Kohoneo  layer  on  the  input  whose  output  nodes  would  feed  the  inputs  of 
a  MFB  network  (Haung,  1987:1-10).  In  his  approach,  the  Kohonen  map  could  be 
used  to  organize  data  which  would  feed  a  MFB  model  input  layer.  This  approach 
could  prevent  having  to  train  the  MFB  to  dassify  data  in  complex  decision  spaces 
and  significantly  reduce  training  convergence  time. 

To  explore  the  possibility  of  combining  different  types  of  neural  networks,  the 
sonar  classification  work  of  Sejnowski  should  be  considered  (Sejnowski,1987:75-89). 
His  work  is  important  because  it  is  one  ^  the  first  efforts,  along  with  Huang,  to  un¬ 
derstand  exactly  what  is  happening  in  the  hidden  weights  and  nodes  of  the  multilayer 
MFB.  In  his  experiment,  Sejnowski  trained  a  neural  network  to  differentiate  between 
the  sonar  return  of  an  underwata  metal  cylinder  and  a  rock  with  a  similar  shape. 
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The  network  uses  two  layers.  The  classification  was  based  on  the  low  order  Fourier 
components  of  the  return  signal.  His  goal  was  to  determine  the  internal  strategy 
used  by  the  network  to  make  a  classification.  A  complete  description  of  the  result  is 
beyond  the  scope  of  this  discussion,  refer  to  the  Sejnowski  paper  (Sejnow8ki,1987) 
for  a  complete  analysis.  He  did  note  that  the  overall  strategy  of  the  net  was  to 
default  to  a  cylinder  response  and  to  detect  the  presence  of  rock  characteristics.  The 
scheme  caused  deactivation  of  all  nodes,  while  the  activation  of  only  a  single  node 
caused  the  firing  of  the  rock  node.  His  work  is  significant,  not  only  for  the  results  of 
the  experiment,  but  the  tools  and  procedures  he  developed  to  characterize  the  inner 
workings  of  the  hidden  layers. 

Huang,  with  a  simple  constructive  proof,  showed  for  several  cases  that  disjoint 
decision  regions  could  be  formed,  not  <Hily  with  a  two  hidden  layer  arrangement,  but 
also  with  a  single  hidden  layer.  He  went  on  to  show  that  although  these  types  of 
complex  decision  regions  could  be  constructed  on  pnper,  in  practice  back  propagation 
training  would  not  converge  in  any  reasonable  amount  of  time  without  the  second 
hidden  layer. 

With  this  analysis,  Huang  proposed  three  alternate  classifiers,  a  fixed  weight 
classifier,  a  hypercube  classifier  and  a  feature  map  classifier.  Fixed  weight  classifiers 
attempt  to  reduce  training  time  by  only  adapting  weights  between  the  upper  layers  of 
the  network.  By  fixing  the  weights  at  some  arbitrary  values,  hyperplanes  formed  at 
the  input  may  be  sufficient  for  the  upper  layer  to  classify  without  additional  training 
of  the  lower  levris.  The  weights  were  fixed  and  tested  in  two  manners,  first  randomly 
set  weights  between  negative  0.5  and  positive  0.5,  and  second,  using  statistics  of  the 
input  data  to  form  grid  lines.  This  solution  proved  inadequate,  as  the  possibility  of 
convergence  depended  on  the  starting  values  for  the  randomly  selected  weights. 

The  second  proposed  solution  was  to  fix  both  of  the  lower  levri  weights  at 
specific  hypercubes  calculated  from  the  statistics  of  the  input  data  set.  The  upper 
layer  of  weights  are  trained  using  back  propagation  or  some  similar  degeneration  of 
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back  propag&tioa  base*.’  on  the  relaxed  requirements.  The  results  were  good,  but 
the  computational  effor.  of  estimating  the  initial  hypercubes  was  significant.  Better 
results  were  achieved  using  the  feature  nup  approach. 

The  feature  map  classifier  is  a  simplification  of  the  combination  neural  network. 
The  first  layer  of  the  network  forms  a  feature  map  using  a  self-organizing  clustering 
algorithm  as  described  by  Kohonen  (Kohonen,ld87).  Weights  to  the  Kohonen  map 
are  trained  unsupervised  allowing  first  layer  feature  nodes  to  sample  the  input  space 
with  a  node  density  proportional  to  the  combined  probability  density  of  all  classes. 
The  first  la^'er  feature  map  nodes  perform  a  function  similar  to  that  of  second  layer 
hypercube  nodes  from  the  previous  example,  except  the  results  are  more  general. 
Each  node  will  go  high  for  a  general  region  of  the  feature  map.  The  upper  layer  will 
do  the  necessary  “and-ing”  and  “or-ing”  of  the  decision  space  for  proper  classification. 

Huang  used  quantized  speedi  vectors  for  his  data  set.  For  image  target  classi¬ 
fication,  the  decision  regions  may  be  complex  enough  to  require  the  additional  layers 
of  a  MFB.  Huang  uses  only  a  single  layer  above  the  feature  map. 

The  next  section  discusses  an  alternate  approach  to  classification  called  the 
counterpropagation  network.  Counterpropagation  is  used  as  the  starting  point  and 
benchmark  for  development  of  the  hybrid  network. 

£.Jt  Counterpropagation 

The  counterpropagalisn  network  b  an  architecture  which  combines  the  self¬ 
organization  feature  of  Kohonen  with  the  outstar  structure  suggested  by  Grossberg. 

>  The  combination  network  is  a  multilayer  feedforward  network  with  a  Kohonen 

organizer  on  the  first  layer  and  a  Grossberg  outstar  on  the  second.  It  has  two 
additional  layers  used  to  train  the  network  which  will  not  be  discussed,  except  to 
^  note  that  a  counterpropagation  net  will  allow  regeneration  of  a  input  vector  by 

specifying  an  output,  henc**  the  name  coiuterpropagation. 


Donald  Woods  (Woods, 1987:473),  noting  two  aberrations  to  the  learning,  sug¬ 
gested  that  a  conscience  be  to  added  to  counterpropagation.  The  first  problem  with 
the  Hecht-Nielsen  algorithm  is  that  the  input  vector  must  be  nonnalized.  limiting 
all  exemplars  to  the  unit  hypersphere.  Consequently,  no  differentiation  can  be  made 
between  vectors  which  are  related  by  a  scaling  factor.  The  problem  can  be  overcome 
somewhat  by  a  training  rule  which  tries  to  minimize  the  distance  measurement  in¬ 
stead  of  maximizing  correlations.  In  this  way  both  (1,1),  and  (2,2)  will  be  mapped 
uniquely  in  the  decision  space. 

The  result  obtained  by  using  Kohonen  type  mapping  as  an  input  another  type 
of  network  is  the  generation  of  an  intermediate,  unclassified  representation  of  the 
input.  The  complexity  of  the  data  to  the  output  layer  is  greatly  reduced  and  the 
construction  of  the  output  net  is  better  understood.  For  backpropagation  using  the 
three  to  one  first  hidden  layer  nodes  to  input  nodes,  will  probably  converge,  and 
network  pruning  can  reduce  complexity  even  naore. 

The  difiiculty  of  this  approach  is  that  the  intermediate  representation  of  the 
data  may  not  be  a  good  vector  quantization  of  the  input.  Convergence  is  guaranteed 
under  the  simple  criterion  that  the  number  of  elements  in  the  Kohonen  layer  is  greater 
than  the  number  of  distinct  decision  regions  in  the  input  data,  ageun  something 
unknown  a  priori.  The  number  is  bounded  by  the  number  of  exemplar  vectors  used 
for  training.  As  the  number  of  Kohonen  units  approaches  that  number,  the  solution 
will  degenerate  to  a  simple  table  look-up  without  generalization. 

On  the  other  hand,  as  the  number  of  nodes  is  reduced,  each  node  represents  the 
average  of  a  number  of  exemplars.  Classification  is  simply  a  matter  of  mapping  the 
Kohonen  map  to  an  output  map.  For  something  as  simple  as  the  counterpropagation 
rule,  the  Grossberg  outstar  format  is  sufficient.  The  output  is  simply  a  binary  word 
one  bit  per  class. 


2.  IS  Summary 

Computation  based  on  neurophysiological  models  has  been  considered  for  a 
few  thousand  years,  though  actual  neural  computers  have  only  existed  for  only  a  few 
decades.  While  the  von  Neumann  computer  remains  the  nxMt  widely  used  computer 
architecture  today,  neural  computers  may  someday  perform  tasks  which  today’s 
computer  cannot. 

Neural  computers  do  not  find  solutions  to  problems  based  on  systematic,  exact 
calculations,  but  make  judgments  or  estimates  using  a  massively  parallel  approach. 
The  approach  uses  from  several  to  thousands  of  individual  processing  elements. 

The  processing  elements,  or  nodes,  are  based  partially  on  how  the  brain  is 
thought  to  function.  While  the  nodes  may  in  some  ways  resemble  the  architecture 
of  a  neuron,  how  this  architecture  with  interconnections  between  hundreds  of  nodes 
is  able  to  solve  real  problems,  remains  a  mystery. 

Many  different  interconnection  schemes  have  been  suggested  to  transform  a 
number  of  nodes  into  a  computing  machine  capable  of  making  judgments.  Each  of 
these  schemes  or  topologies  reveal  their  own  strengths  or  weaknesses  depending  on 
the  type  of  problem  being  considered. 

The  statistical  makeup  of  input  data  seems  to  determines  wkich  type  of  neural 
network  is  best  suited  to  solve  a  particular  problem.  For  simple  decision  region 
problems  with  no  disconnected  or  disjoint  learning  regions,  neural  network  training 
is  easy  and  only  a  few  nodes  are  required.  As  the  number  of  disjoint  or  ambiguous 
regions  increase,  the  size  of  the  network  required  for  convergence  seems  to  increase 
dramatically. 

Classification  problems  would  be  simpler  if  the  input  data  could  be  mapped 
into  a  decision  space  which  lacks  these  disjoint  regions.  Data  self>organization  as 
suggested  by  Kohonen,  may  provide  such  a  mapping.  Combining  Kohonen  self- 
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orgaauation  with  back  propagation  of  error  may  reduce  the  complexity  of  the  deci¬ 
sion  regions  and  consequently  reduce  training  times. 

In  order  to  construct  a  combination  MFB  and  self-organizing  neural  net,  an 
environment  to  analyze  the  dynamic  nature  of  neural  nets  is  developed  for  use  on  a 
color  graphics  workstation.  The  environment  will  allow  the  study  of  several  types  of 
neural  nets  with  application  to  both  measured  data  (the  Ruck  and  Roggerman  sets) 
and  calculated  data.  The  next  section  will  discius  the  organization  of  the  software 
environment  use  to  develop  the  Hybrid  network. 


III.  The  Neural  Network  Analysis  Environment 


In  order  to  study  the  dynamic  nature  of  neural  networks,  an  environment  was 
devised  to  present  the  vast  amount  of  data  in  such  a  way  that  would  allow  insight  into 
the  internal  workings  of  the  net.  A  system  was  designed  for  the  Silicon  Graphics 
nUS  workstation  and  written  in  the  “C”  programming  language.  The  IRIS  was 
selected  over  similar  work  stations  because  of  the  larger  color  table  avulable  than 
on  the  Sun  or  GPX  workstations. 

The  design  method  was  iterative.  A  number  of  working  prototypes  were  de> 
veloped,  adjusting  displays  and  menus  as  needed.  As  more  was  learned  about  each 
topology  and  algorithm,  new  reruirements  were  added.  Consequently,  the  require¬ 
ments  analysis  phase  lasted  through  most  of  the  project.  One  of  the  goals  of  the 
project  was  to  study  different  network  topologies.  Only  as  the  models  were  imple¬ 
ment^  and  tested  did  it  become  apparent  what  type  of  information  needed  to  be 
displayed.  After  a  nundter  of  prototypes  were  developed,  patterns  began  to  emerge 
as  to  which  software  modules  were  problem  specific  and  which  could  be  reused  to 
create  more  varied  and  powerful  modules. 

Special  features  and  displays  were  added  to  the  environment  as  research  into 
the  problems  continued.  The  simple  models  showed  what  might  be  useful.  The 
Anal  product  took  the  basic  designs  and  fit  the  pieces  into  general  purpose  reusable 
components  for  construction  of  new  topologies. 

The  NeunlGraphics  software  package  developed  as  part  of  the  research,  con- 
nsta  of  independent  programs  run  from  a  common  menu.  The  programs  are  selected 
from  a  shdl  or  script  program  which  calls  the  individual  programs.  Each  program 
is  mdependent  and  is  run  as  an  execution  file  from  the  script  program. 


Although  each  program  is  iadependent,  they  all  share  a  common  software  de¬ 
sign.  This  section  will  discuss  the  modules  which  make  up  each  of  the  four  types  of 
neural  networks  which  make  up  the  NeuralGraphics  package. 

S.l  Softvan  Design 

Each  of  the  neural  networks  is  composed  of  a  number  of  independent  software 
modules.  Changiug  the  topology  required  only  small  local  changes  within  a  particular 
module  to  change  the  old  network  to  include  new  training  rules  or  display  features. 
In  addition  to  the  basic  modules,  a  graphics  tool  box  is  provided,  which  is  specific 
to  the  Silicon  Graphics  IRIS  video  display. 

At  the  highest  level  of  abstraction,  s  neural  network  will  consist  of:  a  routine 
to  compose  input  vectors,  a  propagation  algorithm  to  feed  the  vector  through  the 
network,  a  training  routine  for  modification  of  weights,  a  graphic  display  package 
and  an  analysis  routine  for  periodic  testing.  Important  lower  level  modules  include 
the  initialization  procedure  and  an  event  driven  menu  to  control  the  training  and 
operation  of  the  network. 

S.£  Initialization  Module 

The  initialization  routine  has  three  functions.  The  roost  important  is  the  es¬ 
tablishment  of  the  network  in  memory.  In  addition,  the  weights  and  thresholds  must 
be  filled  ather  by  a  random  number  geiterator,  or  a  stored  file  from  a  previously 
trained  net.  The  second  feature  is  to  check  which  software  switches  are  being  in¬ 
voked.  These  include  display  options,  and  training  parameters.  The  third  function  is 
the  equipment  check  to  determine  the  nature  of  the  graphic  displays  and  to  initialize 
the  video  drivers  as  necessary.  The  size  of  the  screen  will  determine  exactly  how 
much  data  can  actually  be  displayed. 


S.2.1  Initialization  and  Dtelamtion  of  the  Net  A  ”C"  data  structure  is  used 
to  defiae  the  network.  Each  t3fpe  of  net  uses  a  similar  structure.  An  array  of  numbers 
for  the  output  and  input  form  the  basic  net.  Between  layers  lie  similar  sirrays  for 
the  various  hidden  nodes,  if  any.  In  addition  to  the  nodes,  there  are  the  weights 
connecting  the  nodes.  These  are  represented  as  two  dimensionaJ  matrices.  Another 
element  that  makes  up  a  neural  net  is  called  a  threshold  value.  Although  similar 
to  a  weight,  these  values  are  stored  as  a  single  dimension  array  associated  with  a 
particular  node  array. 

These  data  structures,  along  witb  a  definition  for  input,  output  and  hidden 
layer  array  lengths  are  stored  in  a  single  program  module  (a  header  file  for  C  pro¬ 
grammers).  The  data  structure  is  declared  and  is  one  of  the  few  global  variables  for 
the  whole  program.  The  net  weights  are  initialized  in  one  of  two  manners:  either 
by  randomly  selected  weights,  or  retrieved  from  a  file.  After  the  data  structures  are 
initialize  the  video  hardware  must  be  initialized. 

S.S.e  Initialization  of  the  Video  Display  The  current  version  of  the  package 
initializes  the  hardware  for  a  Silicon  Graphic  IRIS  workstation.  This  is  a  Motorola 
68020  based  computer  with  an  extended  graphics  capability.  Wherever  possible, 
hardware  specific  commands  are  isolated  to  the  graphics  module.  Porting  the  code 
to  other  types  of  hardware  requires  only  a  substitution  of  the  gl.h  and  device.h 
header  files.  These  device  specific  files  are  provided  by  the  IRIS  compiler  package.  A 
substitute  file  would  include  macro  redefinitions  for  the  hardware  specific  commands. 
Once  initialized,  a  framework  is  established  for  solving  a  classification  problem.  The 
ouin  program  loop  continues  the  process  by  propagating  the  input  to  the  output, 
adjusting  the  weights,  and  checking  the  progress.  The  next  section  discusses  the 
structure  of  that  process. 
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S.S  The  Main  Program  Loop 

The  main  program  loop  consists  of  the  MAKEINPUT  module,  the  PROPA¬ 
GATE  module,  the  TRAINNET  module,  the  TESTNET  and  the  DISPLAY  ana 
SHOW  modules.  A  counter  is  used  to  inhibit  the  calling  of  some  modules  through 
the  loop  to  reduce  computations  effort.  For  example  the  screen  may  be  updated  only 
on  every  tenth  cycle.  However,  each  pass  through  the  loop  represents  one  training 
cycle,  so  MAKEINPUT,  PROPAGATE,  and  TRAINNET  wiU  always  be  called. 

3.3. 1  Makeinput  Essential  to  the  training  of  a  neural  net  is  a  set  of  input 
patterns  and  a  defined  classification,  in  other  words,  an  input  vector  and  a  desired 
output  (doft).  The  function  will  fill  the  net  data  structure  with  a  randomly  selected 
exemplar  from  the  exemplar  or  test  set.  The  module  allows  for  calculating  the 
vectors  or  taking  them  from  a  stored  data  set  read  in  during  initialization.  Also, 
the  initialization  data  preprocessor  may  add  gaussian  noise  to  vectors.  The  work  of 
Sietsma  and  Dow  indicates  that  adding  noise  distributed  across  the  actual  statistics 
of  the  data  seems  to  improve  the  performance  of  the  netw'>rk.(Sietsma,1988:325). 

3.3.2  File  Input  of  Exemplar  Seta  Most  problems  can  be  described  in  terms  of 
a  set  of  input  vectors  and  a  prescribed  classification.  The  NeuralGraphics  software 
requests  a  file  name  on  initialization  which  is  used  to  fill  a  pool  of  exemplars.  An 
exemplar  is  selected  randomly  from  the  pool  whenever  the  makeinput  routine  is 
called.  Also,  the  routine  ensures  that  in  addition  to  random  selection  on  exemplar 
number,  there  is  also  random  selection  based  on  class  type.  This  prevents  excessive 
training  on  a  single  class,  when  the  classes  are  not  evenly  distributed  in  the  input 
file. 


3.3.3  Measured  Data  With  measured  data,  the  data  is  contained  in  a  data 
file.  These  files  can  contain  any  type  of  information.  In  general  these  input  files 
contain  a  data  set  which  exercise  a  particular  area  of  investigation  or  data  that  has 
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been  measured  by  a  real  system.  The  Ruck  data  is  a  collection  of  Zemike  moments 
(Bom  and  Wolf,1964)  calculated  from  laser  radar  images  of  tanks  and  trucks.  The 
file  could  just  as  easily  have  contained  data  &om  frequency  analysis  of  speech  or 
spatial  frequencies  of  character  data.  A  neural  net  should  treat  any  tjqie  data  in  a 
similar  manner.  The  file  also  must  contain  header  information  relating  to  the  length 
of  the  input  and  output  vectors  and  number  of  exemplars  and  classes.  The  data 
is  composed  of  an  exemplar  number,  the  vector  quantizations  and  finally  the  class 
identification. 

5.5.4  GenertU  Purpose  Functions  The  MAKEINPUT  package  contains  other 
modules  necessary  for  handling  the  data  files.  The  three  principle  functions  contained 
in  the  package  allow  for  initializing  the  data  storage,  returning  a  random  exemplar 
to  the  main  routine,  and  statistical  processing  of  the  data.  The  statistical  function 
is  a  stand  alone  program.  By  calling  this  program  from  the  menu,  the  analysis  is 
completed  and  the  program  terminates.  The  statistical  package  performs  a  K  nearest 
neighbor  analysis  of  the  data  and  displays  the  result. 

3.3.5  The  Fourier  filter  problem  A  special  subroutine  for  the  backpropag- 
tion  network  demonstrates  mathematically  calculated  input  vectors.  This  routine  is 
contained  in  the  Makeinput  package.  It  is  not  intended  to  be  part  of  the  software 
package,  but  serves  as  a  template  for  problems  that  require  calculated  exemplars 
rather  than  those  taken  from  a  file. 

The  MAKEINPUT  routine  for  this  problem  computes  an  input  vector  based 
on  the  sum  of  three  sampled  sine  wa\'es  with  an  amplitude  of  one.  The  length  of  the 
input  vector  is  arbitrary,  but  once  defined  in  the  header  information,  represents  the 
Nyquist  sampling  limit.  The  output  vector  has  the  same  length  as  the  input  vector. 
The  output  vector  represent?  the  desired  output  in  terms  of  harmonic  components. 
For  example:  if  the  length  of  the  two  vectors  is  chosen  to  be  eight,  the  input  vector 
is  formed  by  taking  eight  equally  spaced  samples  of  any  of  three  possible  sine  waves, 
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3tn(0),  3m(i),  3tn(2<) . . .  sin(nt).  See  figure  3. 

There  axe  n  outputs,  one  for  each  possible  frequency  component.  The  output 
vector  represents  the  presences  or  absence  of  a  specific  fourier  component. 

In  addition,  a  random  phase  is  added  to  each  of  the  three  components.  The 
output  is  specified  by  the  three  selected  sine  waves.  If  a  particular  sine  wave  is 
selected,  that  output  value  is  one,  if  not  selected  as  one  of  the  three,  its  output 
node  value  is  zero.  The  hypothesis  that  a  mapping  from  the  input  vector  to  a 
Fourier  analysis  at  the  output  seems  reasonable  considering  that  the  sine  and  cosine 
functions,  together  form  a  basis  set  of  orthogonal  vectors. 

These  types  of  problems  can  be  used  to  test  neural  networks.  One  advantage  of 
neural  networks  is  an  amazing  fault  tolerance.  Experience  has  shown  that  this  fault 
tolerance  extends  to  the  software  as  well.  Consequently,  many  software  errors  only 
show  up  as  a  degradation  in  training  efficiency.  Before  a  new  neural  network  can  be 
used  for  measurements,  it  should  be  tested  against  one  of  these  types  of  standard 
problems.  After  validation  of  the  net  against  a  standard  problem,  the  net  could  be 
used  to  analyze  measured  data. 

S.S.6  PROPAGATE  •  Usinjf  tht  net  With  the  organizatbn  of  the  net  in 
m^ory,  the  net  can  bepn  to  learn  and  classify  targets.  To  use  the  net,  the  rules 
for  propagating  the  data  from  the  input  to  the  output  must  be  specified.  This  is  the 
purpose  of  the  Propagate  package. 

Due  to  the  experimental  nature  of  the  presentation  environment,  a  design  goal 
was  to  reduce  the  amount  of  software  an  experimentor  would  have  to  understand  in 
order  to  alter  the  code  to  suit  a  new  experiment.  Both  PROPAGATE  and  TRAIN- 
NET  are  the  two  modules  which  must  be  the  shortest,  cleanest  and  most  precise 
from  a  software  engineering  point  of  view.  Altering  the  screen  displays  could  require 
a  fair  amount  of  expertise  in  graphics  and  program  interfacing,  but  one  would  not 
expect  to  have  to  change  them  very  often.  The  PROPAGATE  and  TRAINNET 
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Figure  3.  Neural  Network  Fourier  Filter 
Note:  Class  A,B  and  C  represent  three  of  n  possible  sine  waves. 
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packages  would  be  expected  to  change  with  each  new  experiment,  so  a  reasonable 
effort  is  made  to  enhance  the  simplicity  and  documentation  of  the  code. 

5.5.7  TRAISNET  >  Teaching  the  net.  The  TRAINNET  module  specifies  the 
training  algorithm  for  the  network.  Four  versions  of  the  trainnet  were  written,  one  for 
each  type  of  training  rule.  The  package  includes  the  backpropagation  nile  developed 
by  Paul  Werbos  (Werbos,  1974),  as  well  as  Kohonen  type  niles  for  unsupervised 
training,  counterpropagation  by  Hecht-Neilsen  and  hybrid  Propagation  developed  as 
a  result  of  this  thesis  effort. 

5.5.8  CHECKERRORS  -  Checking  performance  of  the  net.  Network  perfor¬ 
mance  is  evaluated  in  two  ways.  First,  training  is  periodically  stopped  and  a  test 
set  is  evaluated.  The  second  method  checks  the  performance  after  every  training 
cycle  against  the  current  training  vector.  For  a  general  evaluation  of  the  neural  net 
performance,  a  set  of  training  data  is  run  through  the  net  without  training  cycles 
between  tests.  For  a  more  specific  analysis,  using  a  data  set  different  than  the  one 
used  for  training,  can  show  the  validity  of  the  feature  set  used  to  classify  the  targets. 
The  CHECKERRORS  routine  allows  this  type  of  checking  mid-process  by  ninning 
a  quick  test  set  through  the  net,  then  measuring  the  performance.  The  actual  test 
set  »  specified  in  the  initialization  routine.  When  the  data  is  read  into  memory  dur¬ 
ing  initialization,  the  first  line  of  the  file  specifies  the  number  of  training  exemplars 
followed  by  the  number  of  test  exempleus.  This  partitioning  of  the  data  allows  the 
CHECKERRORS  routine  to  test  the  net  with  a  set  of  vectors  the  net  hasn’t  seen 
before. 

S.4  Program  control 

The  program  uses  two  devices  for  program  control:  the  mouse  and  the  key¬ 
board.  The  mouse  is  used  where  specification  of  a  particular  node  is  required  and  the 
keyboard  is  used  for  larger  program  control  functions  like  reading  or  writing  weights. 
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3-4. 1  Event  Driven  Menu  Ac  event  driven  menu  is  provided  to  control  house¬ 
keeping  functions  of  the  network.  Event  driven  routines  are  more  efficient  than  using 
a  keyboard  poll.  Event  driven  menus  require  a  hardware  event  to  call  the  menu 
subroutine.  No  device  polling  is  necessary.  The  event,  in  this  case,  is  typing  a 
contr^  C  on  the  keyboard.  A  control  C  activates  a  hardware  interrupt  to  the  kill 
addren  vector.  The  program  has  substituted  the  normal  kill  address  vector  with  the 
menu  address  vector. 

While  the  main  purpose  of  the  menu  is  to  allow  the  user  to  save  and  restore 
weights,  the  menu  also  allows  control  parameters  to  be  changed  while  training  is  in 
progress.  The  menu  display  in  the  textport  offers  a  series  of  selections.  Selection  of 
a  particular  item  will  then  prompt  the  user  of  the  parameters  associated  with  the 
particular  call. 

S.4.t  DtSPLAYNET  The  display  routines  refer  only  to  the  graphics  portions 
of  the  display.  These  include  such  functions  as  drawing  the  networic  weights,  drawing 
the  nodes,  setting  the  colors,  finding  data  ranges,  and  drawing  color  ban.  The  pack¬ 
age  is  split  int<  .wo  types  of  graphic  routines.  The  basic  set  contains  those  functions 
which  are  machine  dependent.  In  general,  a  macro  is  used  when  possible  to  allow  for 
redefinition  to  other  machines.  Those  which  are  not  machine  dependent  are  combi¬ 
nations  of  the  basic  routines  that  are  machine  dependent.  An  example  would  be  the 
color  bar  routine.  These  two  types  of  routines  are  included  in  the  graphics  package. 
The  group  of  routines  which  are  specific  to  a  particular  topology  are  contained  in 
the  display  package.  In  general,  when  the  problem  under  consideration  changes  from 
something  like  a  Kohonen  map  to  a  counterpropagation  model.  The  entire  display 
package  is  replaced.  The  graphics  package  is  used  as  an  include  file  so  the  process 
of  building  a  new  display  is  somewhat  reduced. 

S.4-S  SHO  WNET  The  SHC  WNET  package  indudes  all  the  textport  printing 
and  display  routines.  This  package  includes  routines  for  display  of  a  set  of  weights, 
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output  and  input  values  printed  in  ascii  format  in  the  text  port.  Except  for  test 
results,  the  subroutines  are  not  intended  for  use  in  general  operation  of  the  network. 
These  routines  are  intended  for  use  in  trouble  shooting  when  the  program  has  been 
altered.  The  exception  is  the  showoutput  function.  Periodic  presentation  of  both  the 
desired  and  actual  output  is  useful  in  tracking  the  progress  of  the  net.  In  addition, 
the  enor  information  is  displayed  in  the  same  oaodule.  Adding  more  information 
during  the  SHOW  subroutine  is  a  matter  of  changing  the  ‘I” codes  in  line  10  of  the 
SHOW  subroutine. 

3.5  Summary 

The  material  in  this  chapter  outlined  the  major  software  modules  which  make 
up  the  NeuralGraphict  environment.  Although  the  package  is  self  sufficient  and 
can  be  applied  to  almost  any  segmentation  or  vector  quantisation  problem  without 
code  modifications,  the  nature  of  software  dictates  that  sometime  changes  will  be 
required.  The  information  presented,  together  with  the  code  and  comments,  should 
be  sufficient  for  making  changes  to  suit  a  particular  problem. 

The  next  section  chapter  two  purposes:  to  document  validations  and  testing 
the  system  and  more  important,  is  to  answer  the  types  of  questions  the  system  was 
devdoped  to  investigate. 

The  NeuralGraphics  study  environment  is  designed  to  function  as  a  window 
into  the  internal  structures  of  neural  networks.  The  following  chapter  will  describe 
the  observations  which  led  to  the  development  of  a  new  type  of  neural  network,  the 
hybrid  net. 
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IV.  Data  Analysis 


4.1  Introduction 

I 

This  chapter  has  two  purposes:  first,  to  document  the  validation  of  the  software 
and  second,  to  show  how  the  graphics  environment  was  used  to  construct  a  new 
paradigm  for  training  artificial  neural  networks.  The  hybrid  network  is  the  result  of 
examining  learning  characteristic  for  several  types  of  neural  networks,  then  amending 
the  learning  rules  to  correct  several  weakness  discovered.  The  hybrid  network  is 
shown  to  be  more  efficient  than  MFB  for  ambiguous  decision  regions. 

*  Several  types  of  data  are  analyzed  to  validate  the  study  environment.  To 

investigate  the  learning  characteristic  of  MFB,  error  surface  analysis  software  was 
developed.  Using  a  three-dimensional  display,  patterns  of  neuron  weight  adaptations 

I  are  presented  for  simple  neural  networir  problems.  A  Fourier  filter  problem  demon¬ 

strated  the  use  of  the  environment  with  calculated  exemplar  sets.  The  last  two 
problems,  consider  two  seta  of  data  extracted  from  digital  imagery.  The  Ruck  and 

I  Roggemann  data  sets  are  used  to  evaluate  the  features  selected  for  neural  network 

analysis.  The  analysis  of  these  data  sets  is  used  to  contrast  numbers  of  training  cy¬ 
cles  and  accuracy  for  Kohonen  m^,  the  multilayer  perceptron,  counterpropagation, 
and  hybrid  propagation. 

4.£  Error  Surface  Analysis 

This  section  will  discuss  an  experiment  in  error  surface  analysis  used  to  examine 

g 

the  adaptation  of  neural  network  weights  while  training.  Two  methods  of  adapting 
weights  are  examined  and  compared. 

A  neural  network  node  is  the  fundamental  unit  of  an  artificial  neural  network. 

^  It  is  composed  of  an  input  vector,  a  single  output,  and  a  set  of  interconnecting 

weights.  A  node  also  includes  a  rule  for  propagating  the  input  to  the  output,  and  a 

• 


35 


rule  for  training  the  weights.  Three  types  of  nodes  can  process  analog  input  data. 
These  three  types  of  nodes  are  the  back  propagation  nodes,  the  Kohonen  nodes,  and 
similar  to  the  Kohonen  nodes  are  the  counterpropagation  nodes. 

To  Investigate  the  process  of  error  reduction  by  adjusting  the  weight  vector, 
each  type  of  node  will  be  considered  in  turn. 

4.2.1  Error  Reduction  by  Back  Propagation  of  Errors  Back  propagation  of 
errors  was  devdoped  by  Paul  Werbos  (Werboa,1974).  In  recent  work,  a  modification 
to  back  propagation  has  been  suggested  by  David  Parker  (Parker,1986)  called  second 
order  learning.  To  understand  hew  these  algorithms  reduce  error,  a  visualization  tool 
is  included  with  the  package  to  track  the  error  as  the  weights  adapt  toward  a  solution. 

4.2.2  A  Simple  dassifieation  Problem  The  first  artificial  neural  network  sys¬ 
tem  considered  is  the  simplest  S3rstem  possible.  The  simplest  problem  associated  with 
a  neural  network  would  be  that  of  a  single  neuron  with  two  weights  attached.  The 
classification  problem  is  to  differentiate  between  two  points  in  the  decision  space. 
Use  of  only  two  weights  will  allow  plotting  of  the  system  error  as  a  function  of  two 
values,  Wi  and  W^.  A  value  of  one  is  used  for  9,  the  threshold. 

Two  points  are  arbitrarily  sheeted:  (2,1)  for  class  one  and  (1,1)  for  class  zero. 
Now,  the  anooimt  of  error  for  any  two  arbitrary  points  can  be  determined  by  the 
relation: 


error’  =  (1  -  /(ii  *  -1-  *  w,))*  -f  (0  -  f{x2  ♦  toj  yj  *  toj))’  (1) 

The  function  /(u;i,tP3)  is  a  limiting  function  as  described  by  Lippmann  (Lipp- 
mann,1987).  This  demonstration  uses  a  sigmoid  fimction  to  ensure  differentiability 
as  required  for  the  back  propagation  rule.  This  function  is  sometime  called  a  squash¬ 
ing  function,  because  the  output  is  squashed  between  zero  and  one.  The  program  is 
run  a  number  of  times  with  the  results  shown  in  figure  5. 
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Figure  4.  Simple  Neuron  Problem 


In  Figure  5  the  red  and  yellow  tracks  show  the  change  in  weights  from  the 
random  starting  point  to  the  final  convergence  point.  Red  areas  on  the  pbt  indicates 
those  areas  with  the  greatest  amount  of  error  and  blue  represents  the  lowest  amount 
of  error.  Weights  are  plotted  with  tei  on  the  «>axis  and  W)  and  the  y-axis.  The  three 
dimensional  plot  shows  height  of  the  error  surface  on  the  s-axis.  The  graph  in  the 
upper  right  hand  corner  exhibits  the  line  found  by  the  neural  network  which  divides 
the  two  decision  regions. 

Two  point  problems  are  of  little  interest  because  of  the  simplicity  of  the  error 
surface.  The  problem  is  made  arbitrarily  complex  by  adding  additional  points.  The 
example  in  Figure  8  considers  six  points  for  classification.  The  equation  of  the  error 
surface  can  be  generalized  to  include  an  arbitrary  number  of  points. 

n 

error(wi,  toj)*  =  -  f{xi  •  u>i  +  y,  *  u?j))*  (2) 

iml 

Considering  weight  values  between  positive  and  negative  two,  an  error  surface 
is  constructed  to  show  the  rdative  error  for  different  values  of  mx  and  w2.  The  results 
are  shown  in  Figure  8. 


Figure  5.  Six  Point  Error  Surface:  Low  EU 

Note:  No  matter  where  the  weigha  start,  they  move  toward  the  lowest  point  on  the 
error  surface.  Note  the  small  step  sixe  in  areas  of  low  error. 

Eta  s  0.3  Momentum  ^  0.0 


Figure  8.  Six  P<Mt  Error  Surfifie:  Large  Momentum 

Note:  The  valuee  offered  the  beet  oompromiee  between  good  movement  in  the  low 
error  akqM  regions  and  yut  lack  the  wfld  swings  of  the  high  slope  regions. 

Eta  w  0.3  Momentum  m  0.8 

The  egqperiment  was  repeated  several  times  using  different  dusters  of  the  exem> 
plar  pmnts  and  randomly  sdected  initial  weight  setting.  Eadi  initial  wei^t  setting 
relates  to  a  new  starting  point  on  the  error  surface.  The  yellow  tracks  in  Figures  8 
and  9  show  the  weight  values  adjusting  from  higber  values  (red  and  green)  into  the 
blue  regions  that  indicate  a  the  lower  error.  Figure  8  shows  the  reduction  of  error 
for  a  Werbos  badr  propagation  rule,  while  Figure  9  shows  the  decent  down  the  error 
surface  ot  a  second  order  algorithno. 

The  error  surface  routine  denoonstrates  how  the  wd^ts  adapt  from  the  random 
starting  point,  to  the  region  with  the  lowest  error.  While  the  second  order  algorithm 


Figure  9.  Second  Order  Algorithm 

Note:  The  error  reduction  is  emoother,  but  ■ometinwe  nuaeee  nitogether  as  noted 
where  the  line  goes  off  the  screen. 

A1  0.2  A2  «  0.0  A3  »  0.0  A4  »  0.1  A5  «  0.05 


•howed  a  smoother  flow  across  the  error  surface,  the  first  order  seems  to  fjid  the  error 
nunima  with  fewer  training  algorithms.  Second  order  adaptation  requires  adjustment 
fat  four  coefficients.  (Piazza,  1988).  Examination  of  the  error  surface  ensures  that 
use  of  first  order  algorithm  is  sufficient  for  the  foUowing  problems.  Of  the  five 
coefficients  necessary  for  second  order  back  propagation,  two  are  also  required  for 
the  first  order  algorithm.  One  is  the  standard  q,  or  training  rate,  and  another  is  a 
term  which  Lippman  caUs  momentum  (Lippinan,1987). 

Momentum  sometimes  has  the  effect  of  decreasing  training  iterations,  while  at 
the  same  time  may  avoid  local  miniinA- 
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4.2.3  Self^Oryanuing  Nodes  No  graphic  toola  woe  devdoped  to  analyze  the 
Kohonen  and  counterpropagation  node.  Their  learning  paradigms  is  (airly  straight 
forward,  with  the  weight  vectors  moving  toward  the  input  vector's.  Neither  of  these 
types  of  nodes  is  capable  of  classification  on  its  own,  each  of  these  nodes  performs  a 
simple  correlation  with  the  input  vector. 

In  the  case  of  the  Kohonen  node,  the  corrdation  is  a  measure  of  the  Euclidian 
distance  between  the  weight  vector  and  input  vector-  The  counterpropagation  node 
uses  a  dot  product  multiplication  to  correl^  between  the  two.  In  either  case,  the 
weight  update  rules  cause  the  weight  vector  to  move  toward  the  input  vector. 

The  output  of  a  Kohonen  node  is  usually  based  on  the  values  of  adjacent 
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nodes.  For  example,  only  the  node  with  the  lowest  distance  or  higL'st  correlation  of 
a  layer  oi  nodes  is  passed  to  the  output  or  next  layer.  The  rest  are  sometimes  set  to 
zero.  The  learning  process  in  self-organizing  nodes  specifies  that  only  one  or  several 
adjacent  nodes  are  adjusted  with  each  training  iteration.  The  effect  is  to  allow  one 
node  to  capture  a  particular  region  in  the  decision  space.  The  weight  vector  for  a 
particular  node  will  be  equal  to  the  average  of  all  the  input  patterns  for  which  it  has 
been  updated. 

The  counterpropagation  node  is  in  most  ways  equivalent  to  the  Kohonen  node. 
The  difference  is  related  to  the  type  of  data  which  is  being  classified.  The  counter- 
propagation  node  expects  normalized  data,  otherwise  the  dot  product  will  be  scaled 
and  the  correlation  of  the  weight  vector  with  the  input  vector  would  not  be  valid. 
All  input  vectors  must  lie  on  the  unit  hypersphere.  When  the  data  is  derived  from 
transducers  or  instruments,  the  measurements  for  similar  patterns  my  b««  uniformly 
scaled.  Normalization  removes  the  effects  of  this  scaling. 

4.t.4  ResuU*  First  order  back  propagation  of  errors  seems  sufficient  for  any 
simple  classification  problon.  Using  a  value  of  0.3  for  7  ud  a  value  of  0.8  for 
momentum  seem  sufficient  for  training.  Higher  values  didn’t  seem  to  make  any 
difference  in  training  times. 

The  smooth  rounded  tracks  of  the  second  order  algorithm  indicate  excellent 
error  correction.  However,  due  to  the  computational  complexity  of  computing  each 
correction,  the  first  order  algorithm  was  selected  over  the  second  order  algorithm. 
Piazza’s  work  with  larger  network  showed  that  although  the  second  order  network 
does  out  perform  first  order  networks,  the  improvement  is  small  and  may  not  justify 
the  additional  computations  (Piazza, 1988). 

With  selection  of  a  training  algorithm,  the  next  section  discusses  an  applica¬ 
tion  of  the  training  routine  to  another  simple  problem,  identification  of  cosines  in  a 
random  signal. 
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4-3  Feedforward  Networks  as  Fourier  Filters 


This  section  will  describe  sn  experiment  in  which  a  beck  propagation  network 
is  used  to  identify  the  presence  of  specific  components  in  a  harmonic  signal.  The 
experiment  was  devised  as  a  simple  prototype  on  which  to  build  the  graphic  displays. 
Although  the  experiment  was  successful,  the  purpose  of  the  program  was  to  identify 
which  elements  of  the  network  should  be  displayed  in  graphics  format. 

The  equation  describing  the  calculation  of  the  digital  Fourier  transform  looks 
like  it  could  be  represented  as  a  description  of  weights  in  a  neural  network. 

w-i 

A’i(Jb)  s  l/N  *«  •  eoa2rknfN  —  jx^sin2xkn/N  (3) 

iubO 

The  output  of  a  specific  node  is  given  by  the  inputs  are  the  z«  values.  With  k 
output  nodes  and  n  input  nodes  a  system  could  be  implemented  to  generate  Fourier 
transforms  by  setting  the  weights  using  the  equation: 

Weight)^  at  cos2Tkn/N  +  jsin2xknlN~  (4) 

To  adapt  Fourier  analysis  to  a  neural  network,  the  problem  is  modified  to  take 
advantage  of  the  strengths  of  neural  networks. 

To  reduce  the  problem  to  a  manageable  sine,  only  the  in-phase  (cosine)  com¬ 
ponents  will  be  considered,  and  not  the  quadrature  components(sine}.  Further,  only 
the  actual  presence  of  a  component  will  be  detected  and  not  the  magnitude  rdated  to 
that  component.  Although  it  appears  conceivable  to  calculate  coefiidents,  it  would 
not  be  a  good  pattern  recognition  problem.  A  problem  consistent  with  the  intent 
of  pattern  recognition  would  simply  indicate  the  presence  of  a  Fourier  component  in 
sufficient  strength  to  fire  an  output  node. 

A  similar  problem,  calculating  quick  approximations  of  periodic  data  is  dis¬ 
cussed  in  detail  in  the  thesis  work  of  James  Straight.  His  work  pays  particular  atten¬ 
tion  to  time  projections  based  on  the  past  history  of  the  input  data.  (Straight, 1988) 
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Figuic  11.  Fourier  Filter  Neurul  Networic 

Note:  The  podtioa  of  the  three  red  output  node*  indicate  the  input  patten  contained 
the  lum  of  nn2t,*in3t,m4isinSt  with  t  between  wro  and  2r. 

The  back  propagation  algorithm  lequiree  the  output  to  be  in  the  form  of  a 
differentiable  eqoaahing  hmeUon.  The  moot  commonly  uaed  is  the  sigmoid  (Lipp- 
mann,1987).  To  generate  magnitudes  at  the  output  would  require  using  a  mapping 
function  to  recover  the  desired  output  from  a  sigmoid  function.  This  complication 
will  be  avoided  by  using  only  unit  valued  Fourier  components  in  the  input,  and  train¬ 
ing  to  only  denied  outputs  of  one  or  sero.  To  train  the  network  to  generate  actual 
components  of  the  Fourier  transform  is  bqyond  the  scope  of  this  exercise 

4‘i-l  Making  the  fnpet  and  Ontpel  Vectors  The  network  consists  of  an  ar¬ 
bitrary  number  of  input  nodes  with  an  equal  number  of  output  nodes.  Several 
variations  in  the  sise  two  hidden  layers  were  tried.  The  dements  of  the  input  vec- 


tor  are  formed  by  taking  time  samples  of  the  sum  of  three  sine  wave.  For  example:  if 
the  network  topology  consisted  of  n  nodes,  three  values  of  n  were  selected  randomly. 
The  three  sine  wave  are  added  together.  The  sine  wave  were  selected  from:  sin  t, 
sin  2t,  sin  3t  . . .  sin  nt.  Time  samples  of  the  composite  waves  are  used  as  the  mput 
vectors. 

Before  time  «*»"pl<ng  an  arbitrary  phase  is  added  to  each  of  the  samples. 
Arbitrary  phase  is  added  to  each  signal  to  ensure  the  internal  representation  of  the 
input  vector  is  related  to  the  Fourier  decomposition.  An  early  experiment  neglected 
adding  a  phase  component.  Consequently,  the  training  effort  was  greatly  reduced, 
which  indicates  that  a  simpler  internal  representation  was  detected  by  the  network. 

The  resulting  signal  (input  vector)  is  fed  as  input  into  the  neural  network.  The 
output  is  calculated  and  compared  to  the  desired  output.  In  this  limited  problem 
three  of  the  output  nodes  should  omtsin  one,  and  N-3  will  contain  zero.  Evaluation 
of  the  system  consisted  of  considering  one  of  the  fundamental  problems  of  neural 
networks,  finding  the  optimum  number  of  nodes. 

It  was  noted  during  analysis  that  the  greater  the  number  of  connections,  the 
more  complex  of  a  problem  it  could  solve.  For  example,  with  no  hidden  layers,  the 
neural  network  would  cmiverge  as  bng  as  the  phase  component  remained  zero  for 
all  components.  By  adding  two  hidden  layers,  random  phase  could  be  added  to  the 
signals. 

4.3.2  RjtnHa  A  nundaer  of  measurements  were  made  with  the  results  shown 
in  Table  1. 

The  experiment  demmistrated  that  neural  networks  can  learn  Fourier  analysis, 
although  no  evidence  exists  to  show  that  the  internal  representation  discovered  by 
the  network  has  anything  to  do  with  Fourier  analysis.  The  data  set  ^or  these  initial 
classification  experiments,  was  statistically  honmgenous  because  each  of  the  n  pos¬ 
sible  Fourier  components  was  selected  with  equal  probability.  Also,  the  amount  of 
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Figure  12.  Neural  Network  Fourier  Filter 

Note:  Thii  topology  wai  ueed  to  repreeent  a  six  input  Fourier  filter.  The  input  is 
the  ttnae  tample  ram  of  tluee  tine  warae  wHh  raadom  phara.  The  three  red  dots  at 
the  output  show  which  ooo^onents  ate  being  detected. 

phase  added  to  each  sine  wave  conyonent  was  selected  randomlljr  using  a  constant 
probability.  Consequently,  an  infinite  nunober  of  possible  combinations  of  ran^m 
phase  and  components  was  posuble  fwming  an  infinite  trehiing  set. 

Real  problems  usually  lack  this  advantage.  Measured  data  tend  to  have  data 
point  dustered  together,  with  indistinct  class  separation  boundaries.  The  next  sec¬ 
tion  introduves  some  of  the  problems  inherent  to  measured  data,  and  an  approach 
to  solving  these  types  of  problems.  This  approadr,  called  self-organisation,  attempts 
to  map  the  input  data  into  a  more  logical  decision  space  reducing  the  complexity  of 
the  dasrification  problem. 
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Table  1.  Fourier  Filter  Nodes  vs  Training  Time 
4.4  Stlf'Organizaiion  of  Data 

Lippmann  and  others  have  shown  that  the  multilayer  peroeptron  approadi  to 
problem  solving  is  to  the  divide  the  decision  region  with  hyperplanes  in  the  first 
layer  then  appropriately  combine  these  regions  with  ani-mg  and  orbing  functions 
to  generate  class  distinctions  in  the  upper  layers.  Self*organisatioo  has  been  used 
for  some  time  in  image  processing  data  reduction  (KncadwalQ,  1987).  Kncsdwsld 
suggests  using  MFB  as  a  self-organisation  structure.  His  algorithm  uses  a  five  layer 
feedforward  network  which  attempts  to  find  a  smaller  internal  representation  of  the 
input  by  using  the  input  vector  as  the  desired  output  then  trains  on  itadf. 

If  there  are  fewer  nodes  in  the  hidden  layers  than  at  the  output,  the  internal 
nodes  are  forced  to  learn  a  set  of  lower  dimensional  features  than  were  given  at 
the  input.  The  reduction  should  be  a  mote  efficient  pattern  dassificatioa  where 
the  reduced  features  can  be  considered  as  some  sort  of  basis  function  set.  If  the  net 
converges,  these  features  preserve  the  distance  metrics  and  can  be  used  to  reconstruct 
the  original  pattern. 

When  the  network  is  presented  a  pattern  which  contains  mote  information 
than  the  storage  c^tability  of  the  net,  the  best  average  match  w  selected  internally 
and  used  to  generate  the  output.  Kucxewski  reports  that  the  reduction  in  dimen¬ 
sionality  has  no  consequences  until  noise  is  added  to  the  input  vectors  in  which  case 
misclassifications  go  up  significantly. 
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For  application  to  the  aassiiication  problem,  the  bottom  two  layers  may  be 
extracted  as  self-organizing  input  layers,  then  used  as  an  input  to  another  network. 

To  test  this  means  of  self-organoation,  a  five-layer  network  was  constructed 
and  tested  using  a  double  disjoint  region  test  set.  After  several  hundred  thousand 
training  iterations,  the  network  had  still  not  converged  to  a  solution.  The  network 
had  difficulty  finding  a  solution  because  of  the  weight  update  algorithm  used  for 
lower  ievds.  As  more  layers  are  added  the  backward  error  propagation  becomes  more 
diluted  with  each  level  removed  from  the  true  error  measured  at  the  output.  Ae  more 
and  more  hidden  layers  ate  added,  training  time  increases  significantly.  When  the 
reason  for  the  failure  was  understood,  this  line  of  investigation  was  dropped. 

Perhaps  a  network  which  trained  from  both  ends  toward  the  center  could  over¬ 
come  the  dilutirm  of  the  error.  Such  a  network  could  use  self-organizing  based 
training  at  the  input  and  backward  erm  propagation  at  the  output. 

Although  the  ^plication  of  MFB  as  a  self-organization  tool  has  been  denwn- 
strated  by  Kuczdwsld,  application  implies  greatly  mcreased  training  times  which 
is  an  unacceptable  liability.  Use  of  Kohonen  maps  to  set  the  weights  for  an  MFB 
network  may  offer  a  better  solution. 

4.4.  J  Kohonen  Self-organization  Lippmann  suggested  that  back  propagation 
of  error:  works  by  using  the  input  layer  to  construct  hyperplanes  to  separate  classes. 
The  upper  levels  appropriately  combine  regions  formed  by  the  hyperplanes  to  dlow 
classification.  To  increase  the  efficiency  of  each  input  node,  it  may  be  possible  to 
construct  these  hyperplanes  based  on  the  statistics  of  the  data.  To  explore  this 
^yp®^^****»  this  section  exaimnes  Kohonen  self-organization. 

4‘4’^  Kohonen  Self-Organization  and  Simvlated  Data  Using  an  early  outline 
of  Kohonen’s  self-organizing  algorithm  (Kohonen,  1987),  a  display  was  designed 
to  evaluate  different  distributions  of  input  data.  First  a  two  dimensional  input 
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is  constructed,  then  a  three  dimensional,  then  Anally  an  n-dimensional  input.  To 
investigate  this  self  organization  of  data  with  respect  to  the  Ruck  and  Roggermanp 
data  seta,  a  Kohonen  map  is  constructed  which  accepts  file  data  as  inputs. 

An  important  aspect  of  biological  neural  function  is  the  ability  to  organize 
neurons  in  response  to  stimulus.  Kandel  and  Schwartz  believe  that  the  placement  of 
neurons  is  orderly  and  reflects  characteristics  of  the  data  being  sensed  (Kandel,1983). 
Kohonen  maps  attempt  to  dupiicate  this  ordering  of  data  without  respect  to  any 
particular  classification  of  data  with  the  self-organizing  feature  map. 

The  environment  includes  a  Kohonen  map  demonstration  designed  for  a  high 
resolution  graphic  display.  The  first  display  demonstrates  self  organization  of  ran¬ 
domly  selected  two  dimensional  data.  The  presentation  allows  for  exhibition  of  the 
weight  vectors,  a  decision  region  map,  and  a  display  of  the  actual  data  point  being 
presented  to  the  neural  net. 

The  user  may  select  any  of  a  number  of  statistically  distribution  patterns  for 
training  the  network  including  gaussian,  random,  chi-squared.  In  addition,  the  data 
may  be  distributed  over  limited  regions  such  as,  squares,  triangles  and  crosses.  The 
net  was  trained  a  number  o.*  times  to  verify  that  the  nodes  is  distributed  over  sta¬ 
tistical  optimum  regions  as  predicted  by  Kohonen  (Kohonen,  1986). 

In  Figure  13,  a  square  bounds  a  region  of  gaussian  distributed  data.  The  input 
distribution  box  shows  a  plot  of  the  points  as  they  are  presented  to  the  net  for 
training.  The  yellow  Kohonen  decision  region  map  shows  bow  the  Kohonen  output 
nodes  are  distributed  across  the  input  data.  Each  intersection  of  lines  represents 
a  Kohonen  node.  The  position  of  the  node  is  set  according  to  the  corresponding 
position  of  the  weight  vector  in  space.  Since  only  two  weights  are  connected  to 
each  node,  the  node  can  be  regarded  as  a  position  in  the  decision  region.  Notice 
how  the  nodes  reflect  a  higher  concentration  in  the  central  region  where  there  has 
been  more  data  presented.  To  facilitate  tracking  data,  nodes  in  one  comer  are 
colored  white,  while  the  remaining  nodes  are  cobred  magenta  (purple  to  everyone 


Figure  13.  Gwunaa  Distribotioa  over  •  Squue 
Note:  Grid  lioe  ere  cloeer  together  in  the  eater  where  more  detohu  been  preeented. 

but  computer  gr^>hici  epeciehsti).  The  color  code  repreaate  w  arbitruy  partition 
into  two  ciaaeee.  In  every  diatributioa,  the  white  aamples  wiQ  repreaat  25  percat 
of  the  input  data. 

Aa  you  can  aee  from  Figure  13,  even  though  the  white  dots  occupy  only  a 
small  region  of  the  input  Tectors  plot,  on  convergace,  they  occupy  one  quarter  of 
the  Kohona  map  nodes. 

This  is  always  the  case  (<mder  ideal  conditions).  The  number  of  nodes  per 
class  is  ptt^riional  to  the  distribution  of  the  classes  within  the  atire  data  sample. 

at  the  Kohona  decision  region  map,  the  yellow  grid  hnes  are  closer 
near  the  cater  than  on  the  fringes.  This  represats  how  the  decision  region  allocates 
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Figure  14.  GeuatUa  Diftribution  over  a  Croat 

Note:  Number  of  white  nodea  ia  about  25  peroeat  juat  at  white  ioput  data  it  about 
25  percent. 

more  hyperplaaea  to  the  regioaa  '.•Here  there  ia  a  higher,  oonoentration  of  data.  Each 
interaection  of  a  grid  line  r.pieaenta  a  Kohooen  node.  Thia  adf  organisation  of  data 
uaed  at  an  input  la]rer  to  another  type  of  network  may  improve  the  performance  of 
a  network  with  aynergiam  between  the  two  layers. 

4-4’^  KoKonen  Self-Organuaiion  and  Measured  Data  When  the  two  dimen* 
aion  problem  ia  extended  to  three  dimenaiona  the  results  began  to  break  down.  If  a 
Kdionen  prq>roccsa  <d  the  input  data  ia  expected  to  be  useful,  then  each  node  most 
beoonoe  a  clear  winner  for  a  particular  class.  With  three  inputs  to  the  Kobonen 
network  the  results  were  mixed. 
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Although,  clustering  of  the  nodes  to  a  distinct  class  was  exhibited,  a  number 
of  nodes  were  isolated  away  from  the  central  group,  f  lu-ther,  dynamic  analysis  of 
the  net  revealed  that  under  training,  nodes  were  changing  from  one  class  to  another. 

When  the  Ruck  data  was  used  to  train  the  net,  the  effects  were  more  se¬ 
vere:  isolated  nodes,  disjoint  groupings,  and  inconsistent  winners.  Obviously,  sdf- 
organization,  though  proven  with  ideal  data  sets,  will  require  some  modification  to 
the  training  algorithm  for  use  with  less  than  perfect  data.  These  deficienaes  are 
also  reported  by  Robert  Hecht-Neilaea  (Hecht-Neil8en,1987)  and  Dan  DeSieno  (De> 
Sieno,1987)  and  othera,who  propose  several  improvement  of  the  training  method. 
The  modifications  were  included  in  later  models  using  Kohonen  self-organization  as 
part  of  the  network.  However,  these  improvements  were  not  added  to  this  particul  a 
demonstration. 

A  later  section  will  disciiss  implementation  of  these  improvements.  The  next 
section  will  return  to  MFB.  While,  sdf  (»ganization  is  proposed  as  a  method  to 
optimize  neural  network  topologies  and  training  complexity,  self-organization  lack  a 
fundamental  capability  necessary  for  pattern  recognition. 

The  next  section  evaluates  MFB  with  respect  to  measured  data,  and  the  fol¬ 
lowing  section  can  discuss  combination  netwwks. 

4-S  Back  Propagation  of  Error 

Kohonen  maps  cannot  perform  classifications,  only  create  a  self-organized  clus¬ 
tering  of  the  input  data.  Additional  effort  is  required  to  relate  these  clusters  to  a 
p^uticular  classification.  In  the  next  section  back  propagation,  used  with  the  Ruck 
and  Roggermann  data,  is  consider  for  the  classification  mechanism.  Back  propa¬ 
gation  will  be  used  to  bridge  the  gap  between  organized  data  and  classified  data. 
This  section  wrill  discuss  two  critical  issues  rdated  to  neural  networks:  numbers  of 
interconnections,  and  training  difficulty. 
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Using  artifidsl  neural  networks  to  solve  pattern  recognition  problems  requires 
a  tradeoff  analysis  between  system  requirements.  For  an  optimum  system,  the  com¬ 
putational  complexity  must  be  minirntwid  while  at  the  same  time  maximizing  the 
classification  accuracy.  Not  only  are  the  two  requirements  usually  mutually  exclu¬ 
sive,  the  relation  between  these  goals  is  not  wdl  understood  and  are  complicated 
even  more  by  real  world  considerations  i.e.  multiplicative  and  additive  noise. 

4-5.1  CJassicttl  Analysia  for  Pattern  Recognition  The  method  of  k-nearest 
neighbors  is  used  for  the  initial  evaluation  of  the  Ruck  and  Roggermann  Data.  Clas¬ 
sical  methods  for  pattern  recognition  dictate  using  an  Euclidian  distance  measure¬ 
ment  to  the  nearest  neighbor  exemplar  pattern.  These  distance  measurements  are 
compared  to  the  k  nearest  vectors  around  the  test  vector  in  hyperspace.  If  the  ma¬ 
jority  of  the  neighbors  are  a  specific  class,  the  exemplar  is  classified  accordingly. 
The  Ruck  and  Roggermann  sets  were  analyzed  using  this  criterion.  The  results  are 
a  measure  of  the  internal  consistency  of  the  data.  Table  4.5.1  shows  the  results. 


Neighbors 

Roggermann  Target 

Roggermaim  Image 

Ruck  Image 

1 

73 

82 

83 

3 

n 

76 

75 

5 

78 

71 

67 

7 

78 

73 

71 

9 

77 

76 

73 

Table  2.  Nearest  Neighbors  Percent  Accurate  Classification 


Table  4.5.1  shows,  for  the  three  data  sets  considered,  what  percent  of  the 
exemplars  would  have  been  classified  correctly  if  the  decisioc  criterion  was  the  class 
of  the  majority  of  neighbors.  Column  one  indicates  the  number  of  nearest  neighbors 
considered. 

Nearest  neighbor  analysis  shows  that  the  data  lacks  internal  consistency.  For 
any  classification  scheme  to  work  well,  the  data  should  be  clustered  in  some  manner. 


56 


The  table  measurements  demonstrate  that  at  least  about  25  perceni  of  the  time,  for 
all  three  data  sets,  the  nearest  neighbor  represents  a  different  class.  This  certainly 
can  affect  neural  network  training.  It  means  that  at  least  one  quarter  of  the  time 
the  exemplar  pattern  conflicts  with  previous  training.  In  effect  the  net  is  being  lied 
to.  The  result  of  such  lying  is  that  training  time  is  increased,  and  the  net  size  must 
be  increased.  To  accommodate  such  irrationality  in  the  data,  the  network  must 
memorize  the  irrational  exemplars  on  a  one  by  one  basis. 

To  test  this  hypothesis  a  double  disjoint  test  set  was  constructed  and  used  to 
train  a  MFB  network.  The  test  set  consisted  of  four,  three-input  vectors  divided 
into  two  output  classes.  The  network  could  solve  this  problem  with  a  2-3  network 
in  about  80  training  iterations.  The  notation  2-3  indicates  two  nodes  in  the  second 
hidden  layer  and  three  in  the  first.  The  notation  relates  to  the  graphic  displays,  with 
the  number  of  nodes  nearest  the  output,  at  the  top  of  the  screen  coming  first. 

The  number  in  the  other  layers,  input  and  output  is  Axed  by  the  problem.  By 
adding  a  fifth  vector  which  was  very  close  to  the  forth  vector  and  defining  the  class 
as  different,  it  takes  a  20-30  network  with  30,000  training  iterations.  By  adding  only 
one  training  vector,  the  complexity  of  the  problem  was  increased  by  a  several  orders 
of  magnitude.  This  is  a  constructed  example  of  an  ambiguous  decision  region.  Am¬ 
biguous  decision  regions  are  the  weak  point  of  back  propagation.  Any  improvement 
to  back  propagation  would  need  to  address  this  problem. 

For  these  reasons,  to  determine  the  size  network  for  a  particular  pattern  recog¬ 
nition  problem,  consideration  must  be  given  to  the  structure  of  the  input  data.  The 
size  of  the  net  is  related  to  the  number  of  separate  clusters  of  data  in  the  decision 
region.  If  the  structure  is  unknown,  the  only  alternative  is  trial  and  error. 

Analysis  of  the  Ruck  and  Roggermann  data  sets  partially  demonstrate  this 
hypothesis.  Simple  back  propagation  networks  were  used  for  ualysis  with  similar 
results.  The  d}rnamic  analysis  tool  was  used  to  identify  exemplars  which  the  network 
had  trouble  learning.  When  these  exemplars  were  removed  from  the  training  set. 
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Note:  The  number  ot  disjoint  regions  in  the 

decision  space  does  not  completely  dictate  the 
number  of  planes  required  for  seperation. 

I  Amhignous  regions 


Note:  For  good  classification  the  net  must  place 
the  division  in  exactly  the  right  spot. 


Figure  16.  Disjoint  and  Ambiguous  Decision  Unions 

Note:  Multiple  disjoint  decision  regions  problems  are  easier  to  solve  than  ambiguous 
dedsioo  region. 
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the  percentage  of  correct  classificatton  became  near  one  hundred  percent. 


For  the  details  of  the  Ruck  and  Roggennann  data  analysis,  see  the  appendix 
A  and  B. 

The  number  of  independent  decision  regions  could  be  determined  using  self* 
organization  methods.  A  single  layer  Kohonen  offers  a  sort  of  nearest  neighbor 
classification  where  each  output  node  classifies  a  particular  decision  region. 
node,  in  fact,  measures  the  distance  between  the  average  of  a  number  of  exemplars 
just  like  a  k  nearest  neighbor  classifier. 

The  result  of  using  Kohonen  type  mapping  as  an  input  to  another  type  of  net¬ 
work  is  an  intermediate,  unclassified  representation  of  the  input.  The  complexity  of 
the  data  to  the  output  layer  is  greatly  reduced  and  the  requirements  for  construction 
of  the  classifying  layers  are  better  understood. 

The  principle  drawback  to  the  generalized  self-organizing  net  is  that  the  un- 
supervised  reduction  of  the  decision  region  may  not  result  in  a  distance  preserving 
mapping.  Such  is  the  case  when  the  input  data  b  greatly  interspersed.  It  seems 
that  examining  the  nund>er  of  nodes  required  for  showing  clear  winners,  that  is  all 
nodes  are  not  winning  for  one  class  then  later  winning  for  a  different  class,  would 
give  insight  into  the  number  required  for  a  supervised  multilayer  perceptron  model. 
Amhigvova  input  data  is  tht  greatest  hinderence  to  MFB  neural  network  training. 

The  next  section  will  discuss  a  method  to  reduce  the  network  size  for  back 
propagation  networks.  This  method  was  suggested  by  Siest.na  (Sie8tma,ir87).  The 
technique  is  called  network  pruning. 

4.5.2  Neural  Network  Pruning  The  initial  weight  setting  affects  delta  rule 
training  as  well.  When  u~ing  random  numbers  for  the  initial  settings,  sometimes  the 
network  will  train  quickly  and  sometimes,  not  so  quickly.  J.  Sietsma  has  suggested 
there  are  additional  consequences  of  poor  initial  settings.  She  feels,  and  experiments 
have  demonstrated,  that  sometimes  m^  ’e  nodes  are  required  to  find  a  solution  using 
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a  given  weight  update  algorithm,  to  a  mapping  problem  than  are  actually  required 
for  a  minimum  node  solution. 

As  a  feedforward  back  propagation  network  truns,  it  appears  that  some  nodes 
contribute  early  on,  pushing  all  the  weights  in  the  general  direction  of  a  solution, 
but  in  the  end  become  uimecessary. 

As  weight  values  apprt>ach  a  solution,  one  of  several  thi:;gs  can  happen  to  a 
node.  First  all  the  wdghts  connected  could  converge  to  zero.  In  another  situation, 
weights  will  line  up  with  another  set  of  weights  causing  redundancy.  This  occurrence 
has  a  reciprocal  as  well;  a  set  of  weights  leaving  a  node  can  duplicate  the  comriiment 
of  another  set  of  weights. 

Analysis  of  the  weights  for  redundancy  ano  impotence  would  be  difficult  to 
automate,  but  the  consequence  of  these  things  could  be  determined  by  dynamic 
analysis  of  node  firings.  Candidates  for  pruning  could  be  determined  by  observing 
their  action  under  training. 

The  easiest  to  spot  is  a  stuck  node.  Any  node  that  fires  all  the  time  regardless 
of  input,  can  be  eliminated  by  adding,  individually,  the  value  of  weights  to  the 
threshold  of  the  node  abrve  it.  The  second  way  is  to  identify  tandem  nodes.  If  two 
nodes  are  acting  in  tandem,  one  can  be  eliminated.  The  converse  is  also  true,  if  two 
node  are  acting  in  compliment  to  each  other,  that  is,  one  is  high  while  the  other  is 
low,  one  can  be  eliminated. 

Experiments  with  node  pruning  have  shown  that  a  large  node  configuration  can 
be  reduced  to  a  very  small  configuration.  Using  the  Ruck  data,  a  two  hidden  layer 
20-26  net  could  consistently  be  reduced  to  a  6-5  net.  While  consistent  convergence 
could  only  be  obtained  on  the  larger  net,  occasionally  convergence  could  be  obtained 
with  as  small  as  a  b-5  net  and  still  retain  100  percent  accuracy  on  the  training  set 
and  74  percent  accuracy  on  the  test  set. 

Node  pruning  demonstrates  that  fewer  nodes  are  needed  for  an  implementa- 
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tion  of  a  neural  net  problem  than  are  actually  necessary  for  training-  Still  while 
pruning  is  useful  for  back  propagation  network  optimization,  it  left  unanswered  the 
basic  question.  How  many  interconnections  are  require  to  solve  a  particular  prob¬ 
lem?  Kung  (Kung,1987)  tries  to  answer  this  question  by  using  algebraic  projection 
analysis. 

The  number  of  hidden  units  per  layer  dictate  the  space  partitioning  separability 
of  the  network.  The  noore  units  per  layer,  the  finer  the  partition  of  the  decision  region. 
The  work  of  Kaczmarz  suggests  that  the  number  of  nodes  required  can  be  calculated 
(Kung,1987).  Unfortunately,  a  priori  knowledge  about  the  decision  space  is  required. 
This  restriction  is  not  acceptable  in  most  real  problems  as  decision  region  topology 
caimot  be  determined  without  statistical  analysis  of  the  data. 

The  solution  may  lie  in  self-organization  techniques  which  map  the  input  data 
into  well  behaved  functioiu.  These  functions  could  take  advantage  of  the  Kaczmarz 
Algebraic  Projection  technique  by  forcing  the  data  into  a  well  behave  decision  space. 
A  candidate  is  the  Kohonen  learning  rule. 

Unfortunately,  as  shown  previously,  the  Kohonen  decision  regions  are  not  al¬ 
ways  wdl  bdiaved.  A  first  effort  to  study  the  integration  of  two  system  is  the 
counterpropagation  network.  The  next  section  discusses  counterpropagation  which 
may  be  nudified  to  make  the  decision  regions  better  behaved. 

4.6  Counterpropagation 

Counterpropagation  networks  (CPN)  are  described  in  an  article  by  Robert 
Hecht-Nielsen  (Hecht-Nielaen,1987).  A  CPN  network  was  implemented  in  the  lab 
without  good  results,  except  for  simple  problems. 

Counterpropagation  is  a  multilayer  feedforward  network  with  a  Kohonen  orga¬ 
nizer  on  the  first  layer  with  a  Grossberg  outstw  on  the  second.  It  has  two  additional 
layers  use  to  train  the  network. 
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Figure  17.  Coaaterpro|Mg*tion 


The  oouiiterpropegstkw  algwithm  uopknwoted  exactly  a*  tuggeeted  by 
Hecht-NeUaea  and  tested  against  the  Rack  and  Roggennann  Data.  Classification 
accuracy  was  significantly  less  that  the  accaraqr  fitoin  a  MFB.  On  the  Ruck  data, 
while  100  percent  classification  was  a  achieved  on  the  training  set,  only  50  percent 
accuracy  was  noted  on  the  test  set.  The  MFB  classified  at  100  percent  training,  and 
74  percent  test.  See  ^pendix  A  and  B. 

As  suggested  by  Donald  Woods  et  al  (Woods,  1988)  two  modifications  were 
made  to  the  K<^nen  layer  of  the  coonterpropagation.  Woods,  Becht-Neilsen  and 
Deseino  all  suggested  consdenoe  be  added  to  counterpropagation  and  Kohonen  m^. 
Deseino  suggested  an  implementation.  (Deseino,  1988).  A  Kohonen  layer  with  a 
conscience  is  the  first  stq>  to  the  hybrid  network. 


4.6.1  Conseitnet  ExperimenU  related  to  the  thesis  effort,  indicate  that  Ko~ 
honen  works  well  for  data  sets  calculated  from  some  types  of  probability  functions. 
For  example,  a  two  dimensional  input  based  on  a  variety  of  probability  density  func¬ 
tions,  including  constants,  Gaussian,  and  chi-squared  were  tested.  By  defining  a 
region  in  space  to  be  a  particular  class,  then  color  coding  the  classes,  data  clustering 
could  be  tracked  by  noting  winning  regions  on  the  Kohonen  map.  ClasMs  seemed  to 
separate  well  for  calculated  data.  As  expected,  the  number  of  nodes  which  won  for 
a  particular  class  was  related  to  number  of  times  the  class  was  used  as  an  exemplar. 
The  results  were  mixed  for  higher  dimensional  data.  Three  and  four  inputs  were 
tried,  and  completely  fell  ^art  for  measured  data  like  the  Ruck  and  Roggemann 
sets. 

Because  of  the  mixed  results,  modifications  to  the  Kohonen  training  were  con¬ 
sidered.  A  few  suggestirms  were  found  in  the  literature.  Rumelhart  and  Zipser 
suggested  two  things.  The  first  was  to  add  a  distance  bias  to  each  processing  el¬ 
ement.  If  a  node  ^rins  a  competition  the  threshold  is  increased  so  the  chance  of 
winning  is  decreased.  The  second  was  to  update  not  only  the  winning  weights  and 
neighbors,  but  the  losing  weights  as  wdl  just  not  as  much. 

Both  fixes  would  tend  to  force  losers,  as  wdl  as  winners,  toward  the  centroid 
the  input  data  vectors.  Duane  D^ieno  suggested  an  degant  algorithm  for  imple¬ 
menting  these  concepts.  He  called  it  a  oonsderxe. 

The  process  requires  adding  a  data  structure  to  the  Kohonen  net.  The  structure 
can  be  implemented  as  an  array  equal  in  size  to  the  array  of  Kohonen  nodes.  For 
discussion  purposes,  these  are  referred  to  as  probability  bias  nodes. 

The  algorithm  impacts  the  training  rule  in  two  places.  First  in  calculation 
of  the  probability  bias  and  second  in  the  calculation  of  the  winning  node  for  the 
Kohonen  competition.  The  actud  update  of  the  weights  rerndns  the  same. 
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The  probability  biu  array  is  filled  according  to  the  formula: 

(5) 

The  constant  B  is  an  adjustment  parameter.  DeSieno  suggests  0.0001.  C  is  a  constant 
which  controls  the  influence  of  the  conscience.  The  larger  C,  is  the  more  probable 
that  all  nodes  would  win  the  competition  an  equal  number  of  times.  A  value  of  one 
was  used  for  these  experiments.  The  rule  to  select  the  wiimer  of  the  competition  is 
modified  to  add  a  bias  term  to  the  distance  before  selecting  the  wirmer.  The  bias 
term  is: 

hj.C(l/JV-p.)  (6) 

where  N  is  the  number  of  nodes  in  the  competitive  or  Kohonen  layer.  The  conscience 
rules  were  implemented  on  a  hybrid  Kohonen-perceptron  net  with  good  results.  After 
implementation,  it  becomes  obvious  why  many  Kohonen  nets  don't  work  well  for 
measured  data.  All  the  data  points  are  closer  to  each  other  than  any  of  the  randomly' 
set  initial  values.  A  handful  ot  nodes  will  dominate  the  training.  The  rest  wiU  never 
be  adjusted. 

The  experiment  offered  insight  into  several  elements  of  Kohonen  training.  One 
consideration  is  the  rdation  of  the  initial  weight  settings.  If  the  initial  weight  settings 
are  orders  of  magnitude  away  from  the  input  vectors,  the  first  vectors  adjusted  will 
dominate  the  training.  In  other  words,  if  the  initial  neighborhood  size  doesn’t  cover 
the  entire  map,  any  vector  not  adjusted  the  first  time  through  will  probably  never 
win  a  competition  again.  If  the  magnitude  of  the  training  vectors  are  all  very  large 
and  the  initial  weights  are  set  small,  (»  the  first  pass  one  weight  will  win  and  there 
after,  always  win. 

Conscience  inserts  a  probability  bias  into  the  distance  comparison  before  the 
winner  is  selected.  Consequently,  all  nodes  should  have  an  equal  probability  of  firing. 

A  difficulty  was  encountered  selecting  the  parameters  B,  and  C.  Work  with  the 
hybrid  model  suggests  that  the  value  suggested  by  DeSieno  is  not  generally  applicable 
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to  all  problrais.  Using  a  value  ten  times  his  value  yielded  better  results.  The  other 
value  C  should  be  fixed  at  one.  DeSieno  actually  uses  two  adjustments  where  only 
one  is  needed.  Either  of  the  two  equations  could  be  used  fcr  conscience  alone.  A 
more  intuitive  approach  would  be  to  set  the  probability  bias  to: 

Pi  »  amnti/COUNT,^  -  \/N  (7) 

counti  is  the  number  of  times  a  node  has  won.  Bannore  arrived  at  a  similar 
equation  independently  (Barmore,  1988),  he  uses  a  win  rate  disparity  of  1.5  before 
the  conscience  inhibits  a  node  from  winning.  When  B  is  eliminated,  and  C  in  the 
second  equation  becomes  a  relation  between  the  excess  wins,  a  small  number  between 
(-1/N)  and  (1/N  -  1),  and  a  weighing  factor  which  aUows  the  conscience  to  kick  in 
sooner.  The  difference  between  the  DeSieno  method  and  that  used  here  is  that  if 
Pi  is  greater  than  an  arbitrary  small  value  the  node  is  remove  from  the  competition 
completely.  The  DeSeino  equation  has  the  effect  of  a  weighted,  conditional  removal 
based  on  the  number  of  times  a  node  has  won. 

Both  equations  work  well,  but  the  second  is  a  little  easier  to  adjust  to  different 
ranges  of  data.  The  results  demonstrate  why  a  counterpropagation  model  has  little 
chance  of  working  outside  of  well  behaved  input  vectors  and  initial  weights  set  to 
represent  a  distribution  similar  to  the  training  set. 

4-S.i  Weaknesses  of  Counterpropagalion  The  counterpropagation  model  has 
three  primary  weakness.  It  lack  conscience;  usually,  only  a  few  nodes  win  the  distance 
competition  for  most  real  data  sets.  Second,  with  normalized  inputs  all  vc>ctors  are 
mapped  to  the  unit  hypersphere.  Finally,  counterpropagation  isn’t  able  to  solve  the 
disjoint  region  problem  well. 

Since  the  input  vectors  must  be  normalized,  exemplars  are  limited  to  the  unit 
hypersphere,  no  differentiation  can  be  made  between  vectors  which  are  related  by  a 
scaling  factor.  The  problem  can  be  overcome  somewhat  by  minimizing  the  distance 


measurement  instead  of  maximizing  coirdations.  In  this  way  both  and  (2,2) 
will  be  mapped  uniquely  in  the  decision  space.  This  difficulty  is  overcome  by  re¬ 
placing  the  dot  product  of  the  input  with  the  wmght  vector,  with  a  simple  distance 
measurement. 

A  third  weakness  is  partially  a  result  of  using  the  Grossberg  outstar  for 
sifying  the  disjoint  regions.  To  solve  a  disjoint  region  problem  the  output  to  the 
classifier  shot'M  have  an  unambiguous  one  or  zero  at  the  output  node.  If  a  Kohonen 
node  sometimes  wins  for  class  a  and  sometimes  for  class  b,  the  outstar  weights  will 
oscillate  between  the  two.  Convergence  is  not  possible.  Using  conscience  can  ensure 
a  more  efficient  Kohonen  layer  to  prevent  ambiguous  nodes,  that  is  nodes  which  fire 
for  either  class. 

4.7  The  Hybrid  Network 

By  tAlring  the  weaknesses  out  of  the  counterpropagation  network  a  more  pow¬ 
erful  network  is  established.  A  conscience  is  added  to  ensure  maximum  efficiency 
of  the  Kohonen  layer.  A  multilayer  perceptron  replaces  the  outstar  layer.  The  final 
step  in  constructing  a  hybrid  net  is  to  improve  the  interface  between  the  Kohonen 
and  perceptron  layers.  The  counterpropagation  model  passes  along  only  one  piece  of 
information  to  the  upper  layers,  the  Kohonen  competition  winner.  The  winner  spec¬ 
ifies  which  set  of  weights  most  closdy  reflects  the  input  vector.  More  information 
may  assist  the  upper  layer  in  making  a  classification.  Hecht-  Neilsen  suggests  passing 
more  information  to  the  upper  layer.  He  suggests  passing  not  only  the  winner,  but 
second  winner,  third  etc.  The  hybrid  network  inherits  the  same  weakness,  only  one 
node  at  a  time  can  pass  information  to  the  multilayer  perceptron  model.  To  improve 
the  performance  of  the  hybrid  net  a  different  interface  should  be  used  to  pass  more 
information  to  the  perce\Hron  layers. 

The  obvious  solution,  to  pass  the  distance  directly,  was  tried  with  only  partial 
success.  For  small  toy  problems,  convergence  was  found  easily.  The  model  was 


Figure  18.  The  Hybrid  Network 

Note:  First  hidden  layer  uses  kohonen  update  rules  for  weight  adjustments  and 
Second  hidden  layer  used  back  propagation. 
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eooatracted  «ad  tested.  For  noitier  daU,  like  the  Roggemann  set,  convergence  ii  not 
pouible  for  more  tban  a  few  eainplee. 


Figure  19.  Hybrid  Propagation  Environment 

Tbeeohitioaie  baaed  on  een^g  a  wei^tedoomposite  of  the  winner*.  In  other 
words,  a  high  number  is  sent  for  the  winner,  a  sightly  lower  number  for  aecmid  and 
so  on.  A  convenient  function  that  performs  this  ccmversion  is  similar  to  the  sigmoid 
function. 

x<.l/(A  +  4.)  (8) 

The  value  s,  is  a  distance  measurement  for  a  particular  node.  A  is  a  scaling  constant 
less  than  one.  A  typical  value  for  A  is  0.05.  Tbis  valne  in^iersely  scales  all  distances 
to  a  number  between  aero  and  twenty.  The  improvement  in  the  performance  of  the 
net  is  dramatic. 
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The  result  of  using  Kohonen  type  mapping  as  an  input  to  another  type  of  net¬ 
work,  is  a  new  intermediate,  unclassified  representation  of  the  input.  The  complexity 
of  the  data  to  the  output  layer  is  greatly  reduced  and  the  construction  of  the  output 
net  is  much  better  understood.  Using  a  rule  of  thumb,  three  first  hidden  layer  nodes 
for  every  Kohonen  node,  the  network  always  converged  in  few  thousand  training 
iterations.  Network  pruning  seems  to  reduce  the  number  of  interconnections  even 
more. 

The  difficulty  of  this  approach  is  the  intermediate  representation  of  the  data 
may  not  be  a  good  vector  quantization  of  the  input.  Because  the  weight  vector 
associated  with  each  Kohonen  node  tends  towards  the  average  of  a  number  of  close 
exemplar,  convergence  is  guaranteed  if  the  number  of  elements  in  the  Kohonen  layer 
is  greater  than  the  number  of  distinct  decision  regions  in  the  input  data.  That  is,  if  all 
the  nodes  are  used.  This  is  something  unknown  a  priori.  The  number  is  bounded  by 
the  number  of  exemplar  vectors  used  for  training.  As  the  number  ot  Kohonen  units 
approaches  the  number  of  exemplars,  the  solution  degenerates  to  a  simple  table  look¬ 
up  without  generalization.  To  test  the  hybrid  network,  it  is  important  to  demonstrate 
exactly  what  type  of  problems  are  better  solved  using  a  new  type  of  network.  At  first 
it  was  considered  that  the  hybrid  net  would  scale  better  for  larger  numbers  of  disjoint 
regions.  This  hypothesis  proved  incorrect.  Back  propagation  scales  vary  welL  The 
reason,  of  course,  is  obvious.  As  the  number  of  disjoint  region  increases,  each  new 
region  can  take  advantage  of  previous  hyperplanes  to  partition  the  decision  region. 
By  appropriately  placing  the  decision  regions,  sometimes  no  more  byperplanes  are 
needed  to  make  the  distinction.  The  real  advantage  was  unexpected.  By  creating 
ambiguous  dedsion  regions,  that  is  placing  two  exemplars  of  dissimilar  classes  close 
together,  hybrid  propagation  performed  better. 

The  first  experiment  involves  using  four  exemplars  with  a  fifth  used  as  a  spoiler. 
The  fifth  exemplar  was  placed  a  small  distance  away  from  the  fourth  exemplar  as 
indicated  by  the  distance  column. 


Figure  20.  The  Hybrid  Network  Teat  Problem 


These  measurements  are  shown  as  five  regions.  In  the  second  test  the  num¬ 
ber  of  spoilers  was  increased  to  four,  consequently  every  decision  region  was  made 
ambiguous. 
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1 
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0.5 
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0.2 
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78 
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5-10 

0.2 
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87 
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0.2 
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85 
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100 
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5 

0.1 

50,000 

86 

10,000 

100 

5-10 

8 

0.5 

18,000 

100 

10,000 

100 

8-15 

8 

0.3 

26,000 

100 

10,000 

100  1 

10-20 

8 

0.3 

18,00u 

100 

10,000 

100 

5-10 

8 

0.1 

50,000 

0 

2,500 

100 

10-20 

8 

0.1 

50,000 

0 

5,500 

100 

15-30 

8 

0.1 

50,000 

0 

5,500 

100 

Table  3.  Back  propagation  vs  Hybrid  Net 


Note:  In  every  category  hybrid  propagation  out  performs  Back  propagation  for  this 
class  of  problem.  The  problem  under  consideration  is  three  input,  two  output  disjoint 
region  problem.  The  first  case  uses  five  exemplars.  Four  are  distinctly  separable  with 
one  ambiguous  exemplar.  The  ambiguous  exemplar  is  the  same  as  a  one  of  the  other 
exemplars,  but  moved  an  arbitrary  distance  away  and  the  class  is  changed.  This 
distance  is  noted  in  the  distance  column.  The  second  case  increased  the  number  of 
ambiguous  decision  region  to  four,  each  exemplar  has  a  very  close  compliment  of  the 
opposite  class. 


4-8  Summary 

The  hybrid  net  seems  to  work  similar  to  a  nearest  neighbor  classifier.  The 
operator  selects  a  number  of  Kohonen  nodes.  The  number  is  an  estimate  of  the 
number  of  decision  regions.  In  practice  a  few  more  are  required  due  to  inefficiencies 
in  the  conscience.  The  Kohonen  layer  training  moves  the  weight  vectors  attached 
to  these  nodes  toward  the  average  of  a  group  of  exemplars.  For  example:  if  five 
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Kohooen  nodes  are  selected,  then  each  of  the  five  nodes  would  represent  an  average 
of  the  group  of  exemplars  in  a  region  around  them. 

The  experiment  is  based  on  the  assu'^.ption  that  only  a  limited  number  of  data 
clusters  exist  in  the  decision  space.  If  the  training  vectors  are  consistent  then  each 
should  lie  within  a  reasonable  distance  of  these  data  clusters.  Given  that  the  number 
of  data  clusters  cannot  exceed  tbe  number  of  training  vectors,  it  seems  reasonable 
to  assume  that  the  complexity  can  be  controlled  by  limiting  tbe  number  of  vectors 
used  foi  training. 

The  question  becomes,  how  well  does  this  limited  set  of  training  vectors  relate 
to  the  entire  ensemble  of  input  vectors.  If  the  training  set  were  perfect,  the  data 
clusters  would  be  identical  to  the  number  of  classes.  The  worst  case  would  be  where 
there  is  no  clustering  of  data  at  all.  In  this  case  the  number  of  mdependent  decision 
regions  is  equal  to  the  number  of  exemplars.  A  typical  case  b  some  number  in 
between. 

The  hybrid  net  expects  one  Kohonen  node  for  each  data  cluster.  Because  of 
the  conscience  training  algorithm,  in  practice  more  nodes  are  required  to  ensure 
good  separation  between  nodes.  The  back  propagation  network  stacked  above  the 
net  assumed  simple  decision  regions,  and  was  set  to  twice  the  Kohonen  la3rer  for  the 
first  hidden  and  equal  to  tbe  Kohonen  for  the  second  hidden  layer.  Experience  has 
shown  that  for  the  simple,  binary  type  output  of  the  Kohonen  layer,  these  values  are 
sufficient  for  convergence.  Actually,  using  a  multilayer  back  propagation  net  for  the 
output  of  the  Kohonen  layer  is  more  than  is  necessary.  For  this  simple  classification 
problem  Hopfield  or  Grossberg  outstar  would  be  more  efficient  and  train  faster.  The 
back  propagation  method  will  allow  continuous  data  as  input  vectors.  This  flexibility 
allows  the  use  of  more  complicated  functions  as  the  interface  between  the  Kohonen 
layer  and  the  back  propagation  layer. 

The  hybrid  Network  offers  a  solution  to  two  difficult  problems  encounter  in 
other  networks.  The  multilayer  perception  requires  long  training  times  when  ambi- 
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guitiea  exist  in  the  decision  regions.  The  counterpropagation  network  requires  less 
training  time  but  is  unable  to  solve  the  disjoint  region  problems  without  efficient 
use  of  the  Kohonen  layer.  The  hybrid  net  offers  a  compromise.  The  hybrid  net  can 
obtain  k  nearest  neighbor  accuracy  in  a  few  tens  of  thousands  of  training  iteration. 
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V.  Recommendation  and  Conclusions 


Developing  the  hybrid  neural  network  embodied  four  separate  aspects  or  phases. 
The  major  effort  involved  the  development  of  an  environment  for  dynamic  analysis 
of  neural  networks.  Using  these  tools,  the  next  phase  involved  the  anrdysis  of  several 
conunonly  used  neural  network  topologies:  Kohonen  maps,  back  propagation,  and 
counterpropagation.  After  gaming  insight  into  how  the  networks  function,  several 
data  seta  were  analyzed.  The  final  step  in  the  process  was  to  take  the  information 
gained  in  the  first  three  parts  of  the  study  and  develop  a  better  learning  process. 
The  results  of  these  efforts  are  the  Hybrid  network  and  the  NeuralGraphics  software 
package. 

5,1  Gnphies  as  an  Analysis  Tool 

Graphical  analysis  of  a  neural  network  under  training  is  useful  in  several  re¬ 
spects.  The  effectiveness  of  a  particular  node  can  be  observed,  tested  or  the  node 
can  even  be  removed.  The  pruning  function  of  the  study  environment  was  used  to 
demonstrate  this.  Weaknesses  in  a  particular  training  method  could  sometimes  be 
detected.  For  example,  graphic  displays  demonstrated  that  the  Hecht- Nielsen  Ko¬ 
honen  layer  update  rules  resulted  in  only  a  portion  of  the  nodes  being  used  at  alL 
Once  detected,  conscience  was  added  and  the  efficiency  of  the  Kohonen  layers  for 
both  counter  and  hybr.d  propagation  was  improved. 

Error  surface  analysis  allc  ed  two  different  weight  update  rules  to  be  compared. 
Progress  of  the  error  reduction  as  a  function  of  training  cycles  could  be  tracked  from 
the  initial  conditions  to  the  Anal  solution.  So  much  could  be  observed  in  the  displays, 
that  it  wasn’t  possible  to  investigate  but  a  portion  of  the  a.iomalies  noticed  during 
observation. 

Spotting  weaknesses  in  training  methods  is  a  useful  application  of  graphics 
to  neural  networks.  The  fact  that  the  Kohonen  nodes  were  not  all  contributing  to 
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a  solution,  was  obvious  with  a  graphic  display.  Early  versions  of  Kohonen  based 
classifiers  showed  that  only  a  small  portion  of  the  nodes  were  consistently  winning 
the  competition.  This  lead  to  the  inclusion  of  conscience  in  the  training  algorithm. 

Examination  of  the  ranges  in  the  weights  lead  to  a  better  understanding  of 
“pathological  networks”,  network  no  longer  capable  of  learning.  When  the  weights 
are  starting  to  grow  without  bound,  no  amount  of  training  will  bring  the  network 
back  toward  convergence,  and  training  could  be  terminated.  Other  anomalies  were 
noted  as  well.  For  example,  it  was  noted  that  generally  the  majority  of  the  weights 
were  zero.  The  first  major  contribution  of  this  research  was  the  graphics  environ¬ 
ment  which  allowed  detection  of  network  deficiencies.  Discovery  of  these  deficiencies 
allowed  new  neural  network  training  paradigms  to  be  designed. 

5.2  Criterion  for  meanring  error 

A  difficulty  was  encountered  in  establishing  a  criterion  for  measuring  the  per¬ 
formance  of  a  neural  network.  When  comparing  the  results  reflected  in  this  effort 
with  results  of  similar  studies,  different  error  criterion  prevvsnted  exact  comparison. 
This  study  uses  three  methods  to  measure  error.  Each  of  the  three  methods  varies 
from  a  very  strict  criterion  to  a  very  loose  measure. 

For  comparisons  of  one  data  set  to  another,  it  seemed  most  convenient  to  use 
a  very  strict  measure  of  error.  A  classification  of  correct  for  a  given  input  vector 
indicated  that  the  difference  between  the  actual  output  and  the  desired  output  was 
below  a  specified  threshold  for  every  output  node.  For  a  non-ambiguous  classification 
a  value  of  0.2  was  used.  For  a  correct,  but  less  definitive  classification,  a  value  of  0.5 

I 

is  used. 

Unfortunately,  i's  the  width  of  the  output  vector  grew,  as  with  the  sine  wave 
problem,  the  less  likely  it  becomes  for  all  nodes  to  classify  correctly  simultaneously. 

I 

Consequently,  measured  success  of  the  network  appears  to  go  down.  For  the  Fourier 
filter  problem,  a  minimal  criterion  is  used.  Percentage  of  error  is  based  on  each 
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independent  decision.  Changing  the  error  criterion  for  the  Fourier  problem  was 
necessary  to  prevent  misinterpretation  of  the  results.  Under  the  formei  method  the 
networks  would  show  only  80  percent  accuracy,  while  identifying  each  individual  sine 
wave  98  percent  of  the  time. 

Using  and  understanding  different  error  criterion  is  important  for  two  reasons. 
First,  for  comparisons  with  other  work,  the  methods  used  must  be  at  leaM  relatable. 
^  Second,  imlike  most  problem  solving  techniques,  the  goal  is  not  to  determine  a  100 

percent  accurate  solution.  The  goal  is  to  arrive  at  a  reasonable  guess  as  quickly  as 
possible.  For  these  reasons,  understanding  the  error  criterion  becomes  central  to 
neural  network  study.  Comparison  of  different  techniques  is  a  matter  of  evaluating 

I 

training  times  and  classification  effort  for  a  solution  that  is  arbitrarily  close  to  100 
percent.  If  the  completely  accurate  solutions  are  necessary,  some  technique  other 
than  neural  networks  may  be  ntore  appropriate.  In  the  context  of  problems  where 
*  absolute  accuracy  is  necessary,  neural  networks  may  be  used  to  limit  the  search  for 

non-neural  network  solutions. 

After  establishing  an  error  criterion  appropriate  for  a  particular  problem,  the 
f  next  step  was  to  construct  a  network  with  the  minimum  computational  effort.  The 

computation  effort  required  for  a  given  problem  is  related  to  the  size  of  the  network 
and  the  number  of  intercoimections. 

5.3  Determining  Network  Size  and  Training  Time 

The  questions:  how  long  should  the  net  be  trained,  and  bow  many  nodes  are 
^  required  for  a  particular  problem  ?  occupied  many  hours  in  this  effort.  The  author’s 

conclusion  is  that  the  answer  these  questions  depends  almost  entirely  on  the  data. 
For  each  problem  considered  here,  the  number  of  disjoint  and  ambiguous  decision 
regions  in  the  data  set  dictated  the  size  of  the  network  required.  Also,  the  results 
seems  to  indicate  that  it  takes  a  lot  fewer  than  previously  thought.  A  single  node 
seems  very  powerful.  As  pointed  out  by  the  Ruck  and  Roggermann  data,  training 
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until  no  changes  are  detected  in  the  test  set  accuracy  may  be  good  enough  for  real 
problems.  It  appears  that  any  reasonable  problem  can  be  solved  in  a  few  thousand 
iterations.  Training  after  that  point  is  an  act  of  memorizing  the  special  cases.  That 
is,  when  a  tank  exemplar  is  presented  to  the  net  for  training  and  nearest  neighbors 
(in  the  decision  space)  are  jeeps,  the  net  must  memorize  what  makes  that  tank 
different  from  all  the  jeeps  around  it.  Such  memorization  does  not  apply  to  real 
world  problems.  Generally,  classification  will  be  made  on  unique  data  taken  directly 
from  a  sensor  system.  Not  only  is  this  memorization  very  expensive  in  training  time, 
but  requires  additional  nodes  to  handle  the  memorization  of  particular  exemplars.  If 
the  function  of  the  network  is  table  Icok-up,  the  additional  resource  may  be  justified. 
But,  if  the  data  set  is  infinite,  as  data  coming  off  the  backplane  of  a  camera  would 
be,  a  net  only  needs  to  learn  generalities.  Two  factors  were  noticed  which  affected 
the  smallest  possible  network,  momentum  and  network  pnming.  A  first  effort  to 
establish  a  minimum  node  size  tended  to  overshoot  the  optimum.  Overestimation 
may  be  a  natural  consequence  of  using  a  biological  model.  With  the  human  brain 
using  hundreds  of  billions  of  neurons  to  solve  problems,  the  temptation  is  ever  present 
to  use  more  than  necessary. 

Network  pruning  is  a  good  example.  The  first  published  analysis  on  the  Ruck 
data  used  a  100'200  network  (Ruck,1987).  Subsequent  efforts  here  reduced  that 
number  to  10-10.  Network  pruning  reduced  that  number  to  6-8.  Using  the  momen¬ 
tum  term,  reasonable  residts  were  obtained  with  a  3-5  net.  With  this  information 
the  Roggermann  data  resulted  in  a  20-50  net  which  pruned  to  an  18-47.  The  authors 
concltision  is  that  pruning  is  most  useful  when  too  many  nodes  are  present.  When 
starting  with  a  near  optimum  number  of  nodes  pruning  is  only  superfically  useful. 
The  momentum  parameter  may  be  just  as  important  as  pruning. 

Most  of  tbe  first  neural  network  modds  did  not  include  a  momentum  factor 
in  back  propagation  algorithm.  The  computational  effort  to  include  momentum  did 
not  seem  a  fair  trade  off  for  decreased  training  cycles.  By  saving  all  previous  weights 
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each  time,  computational  effort  was  predicted  to  increase,  at  a  rate  greater  than  the 
expected  reduction  in  training  time: 

Late  in  this  study,  the  Piazza  work  (Piazza,1988)  suggested  that  nx>mentum 
carried  additional  benefits,  benefits  both  in  accuracy,  and  the  ability  to  jump  over 
local  nunima.  by  smoothing  out  the  error  surface.  With  the  inclusion  of  nwmentum, 
net  size  on  the  Ruck  data  reduced  bom  a  10-10  net  to  a  3-5  net  with  only  a  tiny 
reduction  in  acciiracy  (5  percent  training  data,  none  for  test  data).  No  additional 
benefit  could  be  gained  from  pruning.  Going  from  200  nodes  to  8  suggests  that  each 
hidden  node  may  be  more  powerful  than  previously  thought. 

Each  node  can  be  considered  more  powerful  still,  if  a  node  is  not  expected 
to  memorize  specific  exemplars,  but  only  general  trends  in  all  the  exemplars.  The 
Hybrid  network  is  an  attempt  to  exploit  this  observation. 

The  second  nujor  contribution  of  this  effort  wsc  the  study  of  how  network 
training  and  node  requirements  relates  to  the  problta.  i*nder  consideration.  This 
research  effort  showed  what  types  of  problems  are  difficult  and  how  these  difficulties 
can  be  avoided. 

5.4  Application  of  the  Hybrid  Network 

The  hybrid  network  is  an  extension  of  Kobonen  mapping  and  counter  prop¬ 
agation.  Using  a  simple  distance  metric,  hybrid  propagation  maps  the  exemplar 
patterns  into  a  new  decision  space.  This  new  decision  space  seeira  to  reduce  the 
burden  on  the  back  propagation  classification  layt... 

The  Hybrid  network  required  an  additional  layer  of  hidden  nodes,  so  the  train¬ 
ing  time  increased  some  to  allow  the  Kohonen  layer  to  organize.  Also,  the  first 
hidden  layer  usually  needed  to  be  larger  than  the  first  hidden  layer  in  back  propaga¬ 
tion  alone.  However,  discounting  the  time  to  organize  the  Kohonen  layers,  training 
times  were  reduced  by  a  third  to  a  half  and  the  network  showed  a  greater  ability 
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to  memorize.  This  ability  wu  noted  in  analysis  of  the  Roggermann  target  data 
analysis. 

In  general,  the  Hybrid  net  works  better  than  a  multilayer  perceptron  for  prob¬ 
lems  that  include  ambiguous  decision  regions.  For  small  clearly  distinct  decision 
regions,  the  multilayer  perceptron  seems  to  be  more  efficient  more  efficient. 

The  third  major  contribution  of  this  research  was  the  hybrid  network  which 
outperformed  other  networks  configuration  for  specific  types  of  problems. 

5.4.1  Summary  Of  the  four  neural  networks  considered,  each  had  its  own 
strength  and  weakness.  The  Kohonen  maps  caimot  solve  classifiration  problem 
without  some  interface  to  a  classification  network.  Both  Kohonen  and  counter¬ 
propagation  were  inefficient  without  augmenting  the  weight  update  rules  to  include 
a  consdence.  Even  with  a  conscience,  the  nodes  were  not  one  hundred  p>ercent  effi¬ 
cient,  mapping  one  node  to  a  one  decision  region.  With  a  Multilayer  perceptron,  as 
the  number  of  discoimected  disjoint  decision  regions  increased,  the  number  of  nodes 
required  to  solve  the  problem  increased  in  a  non-linear  fashion. 

The  Hybrid  network  shows  promise  of  bdng  able  to  solve  the  the  ambiguous 
decision  region  problem  of  the  back  propagation  algorithm.  As  dissimilar  data  points 
moved  closer  together,  the  Hybrid  network  the  number  of  nodes  required  seems 
to  increase  linearly.  Also,  the  Hybrid  network  is  a  useful  tool  for  data  analysis. 
Still,  more  work  could  be  done  to  improve  the  efficiency  of  the  Kohonen  layer.  The 
conscience  rule  doesn’t  seems  to  map  exactly  horn  one  decision  region  to  one  kohonen 
node.  This  may  be  possible  by  using  some  other  paradigm  to  train  and  set  the  size 
of  the  Kohonen  layer. 

5.S  Recommendations 

The  Neural-Graphic  study  environment  is  like  a  window  into  a  complex  math¬ 
ematical  process.  For  every  question,  answered  several  more  were  raised.  For  ex- 


ample,  ’t  was  noted  that  generally  the  majority  of  the  weights  were  zero.  Most  of 
the  questions  associated  with  these  types  of  observations  went  unanswered.  Future 
investigation  may  want  to  consider  the  effects  of  removing  near  zero  v2dued  weights. 

In  implementing  neural  network  in  dedicated  processors,  real  world  constraints 
should  be  considered.  An  important  consideration  for  making  integrated  circuits 
would  be  the  constraints  on  the  size  and  accuracy  of  the  weights.  In  a  chip  these 
weights  could  be  implemented  as  resistors.  Further  investigations  should  try  hard 
limiting  the  weights  to  different  ranges.  Also,  the  dynamic  ranges  requirements 
could  be  explored  by  adding  varying  degrees  of  noise  to  the  weight  values.  This 
would  provide  insight  into  how  accurate  the  weight  values  would  have  to  be. 

The  NeuralGraphic  software  package  allows  rapid  study  of  segmentation  and 
vector  quantization  paradigms.  As  many  government  organizations  are  purchasing 
commercial  software  to  perform  these  types  of  task,  making  this  package  available 
could  save  the  Air  Force  thousands  of  doUan.  To  make  this  package  of  commercial 
quality  would  only  require  porting  the  source  code  to  several  types  of  machines. 
Although  NeuralGraphics  was  written  for  a  Silicon  Graphic  IRIS,  with  only  small 
changes  it  could  be  made  to  run  on  a  Sun  workstation,  a  Micro  Vax  III  or  even  a 
mainframe  Vax  (without  the  graphics). 
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Appendix  A.  Ruck  Data  Analysis 


The  Ruck  d&ta  set  U  a  collection  of  52  exemplar  vectors  and  27  test  vectors 
extracted  from  laser  radar  range  imagery.  The  input  vector  features  are  base  on 
Zemeki  moments  which  offer  position,  shift,  and  scale  invariant  to  the  exemplar 
pattern. 

The  network  configuration  consists  of  22  inputs  and  four  outputs.  The  four 
output  indicate  classes  related  to  tanks,  jeep8,trucks  and  POL  tanker  vehicles.  The 
number  of  hidden  units  is  indicated  by  the  column  Netsize  in  each  of  the  tables.  The 
first  algorithm  used  to  evaluate  the  data  set  is  a  multilayer  feedforward  backpropa- 
gation  network,  without  theta  training  and  momentum  equal  0.7.  An  r;  of  0.3  was 
used.  Several  values  between  0.3  and  0.1  were  tested  each  without  any  significant 
difference  in  training  times  or  accuracy.  The  only  difference  noted  was  that  using 
smaller  values  of  17  caused  the  training  to  converge  in  a  smoother  fashion. 

The  training  statistics  are  based  on  1000  random  samples  pulled  from  the 
training  set.  The  test  statistic  are  calculated  by  sequential  classification  of  a  separate 
test  set  and  calculating  error  statistics. 

A  classification  of  right  means  that  for  every  in-class  vectors  a  value  of  0.8  and 
above  was  found  at  the  output  together  with  a  value  of  0.2  and  below  for  each  out 
of  class  node.  A  guess  is  an  indication  of  an  ambiguous  output,  but  still  above  0.5 
for  the  in-class  node  and  below  for  the  out  of  class  node. 

A.l  Backpropagation  Rules 

For  a  base  line  estimation  of  the  number  of  nodes  required,  several  net  config¬ 
urations  were  to  50,000  training  cycles  with  the  results  shown  in  table  4. 

Considering  that  the  Ruck  classification  problem  can  be  solved  with  a  3-5 
sj'stem  (three  in  the  first  hidden  layer  and  five  in  the  second),  improvement  would 


Netsize 

Lower 

Upper 

3 

5 

5 

5 

10 

5 

10 

10 

15 

15 

26 

20 

Training 

Right  Guess 


Guess 


68.2  74.7  100.0  100.0 

69.2  74.2  100.0  100.0 

74.4  76.2  100.0  100.0 


Iterations  I  Converging 


50,000 

50,000 

50,000 

50,000 

50,000 

50,000 


Table  4.  Percent  Accuracy  vs  Net  Size 

Note:  reasonable  good  results  were  obtained  for  the  3-5.  Nets  larger  than  10-10  did 
not  improve  accuracy. 

be  difficult.  Convergence  with  such  a  small  system  indicates  that  the  data  is  well 
behaved  with  only  a  handful  of  disjoint  regions  and  no  ambigous  regions.  If  this  is 
the  case,  improvement  can’t  be  expected  for  either  a  hybrid  network  or  a  counter 
propagation  network. 


A,2  Counter  Propagation 

Counter  propagation  allows  convergence  in  only  a  few  thousand  training  cycles, 
but  the  generalization  properties  are  poor.  Convergence  here  means  100  percent 
accuracy  on  the  training  data.  Although  the  training  set  classifies  at  100  percent, 
the  test  set  shows  only  50  percent  correct  classification.  The  baseline  for  the  test 
data  is  74  p>ercent. 

Probably  the  poor  performance  cannot  be  attributed  directly  to  the  counter 
propagation  paradigm.  The  Hecht-Neilsen  model  for  CPN  does  not  use  the  Kohonen 
nodes  effectively  because  a  conscience  mechanism  is  not  included. 


A.S  Hybrid  Propagation 

Table  6  shows  the  same  tests  using  a  hybrid  network. 
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Kohonen  Nodes 

Training 

Right 

20 

.■i6.3 

30 

56.8 

60 

100.0 

Cycles 


Table  5.  Counler  Propagatioa:Kohonen  Node  vs  accuracy 

Note:  Counter  propagation  demonstrates  very  poor  generalization  with  good  con¬ 
vergence. 


Net  Size 
Kohonen 

First 

10 

5 

15 

10 

20 

10 

30 

15 

60 

30 

Training 

Right  Guess 


Test  Cycles 


Table  6.  Hybrid  Net:  Kohonen  Nodes  vs  Accuracy 

Note;  The  Hybrid  net  compared  to  che  BPN  seems  to  trade  training  time  for  number 
of  nodes  while  retaining  same  accuracy. 

A  4  Summary 

As  expected,  the  simple  backpropagation  rule  outperformed  all  both  counter 
propagation  aud  hybrid  propagation.  While  the  CPN  did  converge,  the  poor  perfor¬ 
mance  on  the  test  set  demonstrated  an  inability  of  the  network  to  generalize.  The 
Hybird  net  classified  data  equally  as  well  as  the  BPN,  yet  more  nodes  were  required 
to  obtain  the  same  performance.  The  advantage  of  the  Hybrid  net  was  a  reduced 
training  time. 
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hardware  OUTPUT:  .Vo»# 

MODULES  CALLED:  Nont 

CALLING  MODULES:  Main  program  Shtll 

AUTHOR:  Grrgorg  L.  Tarr 
HISTORY: 


^include 

•truct  iwt: 

#dcliae  w«jt(A)  iH(«ount%(A))=!=0) 

extent  iut  nx*iui(): 
iut  stoj)it=  lOOOOOO; 

int  connisO.right.good.nu^ai.t^t, display  =  lOO.slioe.wsigliUslOO; 
extent  iut  decisions.rast; 

<^tent  iut  ^xaiii.test.fxainplars; 

inain( ) 

{ 

/*  For  in>liah:aUon  onig  */ 
liidt'.OH*'  =  10; 
liul**.teo  =  10: 

INITIALIZED: 

DISPLAY.NETO; 
wlttks  (count  <  stopit)  { 

/*  Rradiag  the  monat  alloirx  magtntfirafioo  and  nodr  prnniug  V 
rlirck.niouwH ); 

11:.5.I  Nor  30  1988 


'.HI 


3n 


luaiu 


40 


r»o 


Ptkft  t  of  yfnfra/.f 


inaiu(general.c) 


MAKEJNPUT(net.inp,n<ft.clo(l.— 1); 
FEED.FORWABDO; 
C'ME<'K.ERRORS(nvt.oiitp.ii«t.do(l); 
TRAIN.NETO; 


wait(«lisplay){  SIIO\V(); 

ifl«ain.t«t!=0)DO.TEST( ); 
calculate_<"rror( ): 
makr  j[raph(600..470. ); 
DfSPLAY.NETO: 


tRfaal  ==  1  ){DISPLAY.NET();) 

C0UHI+4.; 

display  .rount( ): 

} 


/*  Sapt  leeighti  on  itrmtnaiion  */ 
writ«,s(d,WMghts( ); 


jj  n  ffop  10  mo 


Pngt  2  of  ffotml.c 


(defluitions.h) 


DATE:  II  .4iif  1988 
VERSION:i.l 

NAME:  Difinriloms.k 

MODULE  NUMBER:  I.S 

DESCRIPTION:  SUudar^  DtfinHont  «nd  Macros 

ALGORITHM:  None 

PASSED  VARIABLES:  Nout 

RETURNS:  Nonf 

GLOBAL  VARIABLES  USED:  Nonr 
GLOBAL  VARIABLES  CHANGED:  Nomt 
FILES  READ:  Non* 

FILES  WRITTEN:  Non* 

HARDWARE  INPUT:  Non* 

HARDWARE  OUTPUT:  Non, 

MODULES  CALLED:  Non* 

CALLING  MODULES:  Non* 

UTHOR:  Grtgorj  L.  Tnrr 
HISTORY: 


#<i«!flue  GRAPHIC'S  TRUE 

TABLE  (M 
Bytn.cotor  394 

loo|>i(ON)  for(isO;i<ON;i-f-f ) 

#<leflue  loo|«j(ON)  Rirlis03<ONJ-f-f ) 

loo(>k(ON)  fbr<ksO;k<ON;k-t"»-) 

#<l«fltt«  loopij(ONE.TWO)  for(isO;i<ONE:i++)lbr(j=Og<TWOj++) 
HARD.ON  RED 
INDETERM  (TABLE/3) 

#(k«iM  UARD.OFF  BLUE 
#if  FALSE 
#defliM  TRUE  I 
#*hiAu«  FALSE  0 
output  <1 
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» 


V 


30 


40 


sn 


/*0ff  /  of  irfintiton.n.k 


(deflnitioiM.h) 


input  22 
hiile.one  20 
#<lcilue  liidv.two  20 

lin«  priiit(("\tt'*) 

(iwply  100 

dslafil^  «.  Adat«\nick.d«ta” 
Apring  200 
#dcflii«  nvight.a  20 
vitlco  1.00 


All 


m 


n;f4  iVni.  SO  I98S 


P«ff  i  of  4€finthonM. 


(uet.h) 


• 

*  DATE:  .1  Ortokr  I9S9 

*  VERSIO'f:  t.O 

* 

*  SAME: 

*  MOBI  LE  NVMBER:  t.  J 

*  DESCRIPTION:  Strueiurt  Dffinthon  for  tir  Ntarol  Nti 

*  10 

•  AUTHOR:  Grtfort  L.  Torr 

•  HISTORY: 

• 

#<laflne  oixr  100 

/*  CnIiftI  Drfiniliou  .  inrinory  aUoeaUi  a!  romfilt  limt  */ 

•tract  iicural.net  {  m 

floNt  outp[oi«c]; 

flunt  doft(aiac]; 

Hunt  «2(oiic][«Mc]; 
flout  w3jaoin(m»r}l»ur); 

float  •w2(mfc](MSc]; 

30 

float  t3(M3c]; 
float  outpjiiaik(aitc]; 
float  )2(«xc]; 
float  y2.m«ak(mxc]; 
float  }r2.<it(Mir]: 

to 

float  wl(irixr>](mf<>]; 
float  wljuouijMScltMlc];  - 

float  awl[Mfc](msp]; 

float  tlfaitr]: 

float  yl(«itc]: 

float  yl.nuMk[mxe];  r,o 

float  y  l.cH(in(c]; 


M.Sf  Nor  W  im 


Paff  /  of  Hfl.h 


(net.h) 


flmit  .v0(iiise](8isv]; 
floAt  w0.nioni[M»‘j(ai*r]: 

floAt  awOfsiirjfsit**]; 

floAt  tO(iii«r]: 

m 

fluAt  iup(sii«]: 
floAt  iup.niAtk(*iir]; 
iat 

iut  riaMi.raant()ii*r]; 


)  : 

iut  o«i(pul.hi<l<*.on«.bi«10*'*>-<nput; 


7n 
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Poft  i  »/  ntt.k 


INrnALIZE(imtiaUM.c) 


*  DATE:  J  ABfUit  19S8 

*  VEnSIOS:  I 

* 

•  ?IAME:  l»ihtili:»linn  Mo4bU 

•  MOOVLE  SVMBER:  t-i 

•  DESCRIPTION:  fmihalhatuM  of  Iht  iiopUf  hmrdmor,. 

•  «iif  lajiai  itti  data,  and  art  data  sirariarr.  in 

» 

•  ALGORITHM:  Non* 

•  PASSED  VARIABLES:  Naat 

•  RETURNS:  None 

•  GLOBAL  VARIABLES  USED:  Nti 

•  GLOBAL  VARIABLES  CHANGED:  None 

•  FILES  READ:  Scat 

•  FILES  WRITTEN:  Noat 

•  HARDWARE  INPUT:  Noat 

•  HARDWARE  OUTPUT:  Noat  so 

•  MODULES  CALLED:  Neat 

•  CALLING  MODULES:  Noat 

•  AUTHOR:  Gnport  L.  Tarr 

•  HISTORY: 

• 

#iacin<le  "dafialtlMa.h” 

f|luich«<l«"4««ic« .  It" 

#iacln<l«  <itdio.h> 

#iuclu<l«  <aign«l.h> 

#iachi<k  "■•t.k" 

«xteru  oit  rount; 
nxt«wn  atnict  iminii.iirt 
flout  rMHJoiii<): 


mctmi  iut  nii>iin(): 


»n 


INITIALIZeo 


INITIALIZE 


21:58  Nor  10  1998 


Paft  I  of  laHialhf.c 


INITIALIZC-iuit.n«i(imtialiu.c ) 


{ 

•ign«l(  SKt  I  NT.niraii ): 

!iiitjKmm( ): 
inmu( ); 
iiiit.iirt( ); 

) 

init.iM>l( ) 

{ 

itt«  ij: 

ronnt  =  0: 

loo|>u(sij«.«ur)  ii«t.w2fi][)]  s  raa<loiii(): 
loopi(«se)  n«t.t3(i]  =  rMidom(); 
loop«j(Msrjii«) 

n«i.wlfi][i]  s  rMidoni(); 
kK>{>i<4av)iiet.tt[i)  ss  tsndomO: 

Mt.wOplC]  »  random^; 
lootN(Msr)ii«t.tO(i]  s  nuKtoni<); 
loopi(iHsr)  iiH.)r3.niaak(i]  =  TRl'E: 
loopi(iiiir)  fKi.yl.iniMk[i]  s  TRl'E; 

nH.inp.mMiifi]  s  TRl'E: 
toopi(Mse)  iwi.ouip.mwkp]  s  TRl'E; 

» 

/*  OVarMfc  •  raadam  •■mtrr  tftwre»  —O.S  +  0.5  */ 
fluMt  random  () 

{  flont  x: 


imt..uet 


j/.M  .Vop  JO  tots 


pH  ft  1  of  toitioihf.e 


J 


V 


Appendix  B.  Roggemann  Data  Analysis 

The  Roggenun  datji  set  ootuista  of  two  probiems.  The  lint  is  a  simple  target/non¬ 
target  classification  based  on  statistical  rdations  of  an  input  scene. 

The  second  problem  takes  a  target  identified  from  the  first  part  and  using  an 
expanded  vector  quantisation  of  the  scene  make  a  specific  classification  of  vehicle 
type. 

B.I  Tojgti/Non-'^argti  Claastfieation 

The  input  data  is  a  vector  quantization  of  three  statistical  features  of  the  image 
d^a.  The  first  va^ue  of  the  input  vector  is  related  to  the  ratio  of  the  height  to  width 
of  a  htci  segmented  from  an  infrared  image.  The  second  feature  is  a  ratio  of  the 
energy  in  the  Uoo  to  the  energy  in  the  background.  The  third  number  represents  the 
standard  deviation  of  the  blob  pixel  values. 

The  network  configuration  consists  of  three  inputs  and  two  outputs.  The  two 
outputs  indicate  target  (1,0)  or  a  non-target  (0,1)  with  one  going  high  while  the  other 
goes  low.  The  number  of  hidden  units  is  indicated  by  the  column  Netsize  in  table  7 
f<»  two  hidden  layers.  The  first  algorithm  tested  uses  a  multilayer  feedforward  back 
propagation  network,  without  theta  training  and  without  a  momentum  term.  An  r) 
of  0.3  was  used.  Several  values  between  0.3  and  0.1  were  tested  each  without  any 
significant  difference  in  training  times  or  accuracy.  The  only  difference  noted  was 
that  using  smaller  values  of  ri  caused  the  training  to  converge  in  a  smoother  fashion. 

The  training  set  consisted  of  600  samples.  The  training  statistics  are  based  on 
1000  random  samples  pulled  from  the  training  set.  The  test  statistic  are  calculated 
by  sequential  classification  of  a  separate  test  set  and  calculating  error  statistics. 

A  classification  of  right  means  that  for  every  in-class  vectors  a  value  of  0.8  and 
above  was  found  at  the  output  together  with  a  value  of  0.2  and  below  for  each  out 


NeUize 

Lower  Upper 


Traming 

Right  Guess 


Table  7.  MFB:  Percent  Accuracy  vs  Net  Size 
Note:  The  20-SO  net  seems  to  be  the  most  efficient  and  accurate. 

of  class  node.  A  guess  is  an  indicatioo  of  an  ambiguous  output,  but  still  above  0.5 
for  in  class  node  and  below  for  out  of  dass  node. 

Training  was  extended  for  network  configurations  that  appeared  to  have  the 
best  chance  of  converging.  Training  was  stopped  when  accuracy  rates  appeared 
stable  or  no  longer  converging. 


B.S  Bocktnrd  Propagation  Rate* 

For  a  base  line  estimation  of  the  number  of  nodes  required,  several  net  config¬ 
urations  were  to  50,000  training  cycles  with  the  results  shown  in  table  7. 

Since  the  problem  consisted  of  over  800  exemplars,  as  expected,  a  larger  net  is 
requited.  The  large  size  of  the  data  set  indicates  a  greater  probability  several  disjoint 
regions.  The  fact  that  the  data  never  did  better  than  a  about  80  percent  indicated 
ambiguous  dedsion  regions.  As  expected,  traiiting  time  also  increases  and  to  bring 
the  network  up  to  the  maximum  performance  requires  over  500,000  iterations. 


85 


■S9 

Upper 

Guess 

Test 

Right 

Guess 

Training  Cycles 

59.2 

78.7 

70.0 

100,000 

79.0 

nQ 

58.0 

100,000 

iKl 

67.7 

78.7 

72.0 

Table  8.  MFB:  Extended  Training  of  Converging  Topologies 

Note:  There  is  no  significant  improvement  between  50,000  and  500,000  training 
cycles. 

Using  the  weights  save  from  the  20-50  network,  neural  network  pruning  tech¬ 
niques  were  employed  see  if  any  improvement  could  be  made  in  the  efficiency  or 


accuracy. 


Upp« 

Training 

Right 

Guess 

■mQ] 

Guess 

SO 

87.2 

78.7 

57.0 

70.0 

47 

12 

69.0 

75.7 

73.0 

41 

12 

81.7 

88.7 

41 

11 

74.0 

82.7 

76.0 

41 

11 

74.0 

82.7 

76.0 

38 

7 

59.0 

62.0 

11.0 

Table  9.  Pruning  to  Find  the  Optimum  Size  Network 
Note:  Pruning  the  network  found  a  better  soiution  with  an  11-41  configuration. 


B.a  Counter  Propagation 

Because  of  the  long  training  times  and  large  net  size  require  for  convergence  the 
Counter  Propagation  net  was  used  to  try  and  get  a  measure  how  many  complex  the 
decision  regions  are  present,  counterpropagation  fail  to  classify  with  any  reasonable 
numbers  of  nodes. 


The  experiment  was  halted  after  training  30  samples  on  to  20  kohonen  nodes. 
The  intent  of  the  experiimat  is  to  find  a  less  complex  system  than  the  multilayer 


Samples 

WESi 

ES9 

Guess 

8 

12 

27.0 

48.2 

12 

12 

27.0 

48.2 

20 

12 

89.2 

17.0 

55.6 

16 

100.0 

30.7 

71.0 

■Kl 

20 

100.0 

29.9 

Table  10.  Counter  Propagation:  Kohonen  Node  v«  Samples 

Note:  When  any  more  than  30  exemplars  were  used,  100  percent  accuracy  couldn’t 
be  readied. 


perceptron  aUme.  With  the  20>  40*20  net,  the  number  of  interconnection  for  only  30 
samples  is  greater  than  the  back  propagation  modd. 


B.4  Hykrid  Pivpsgetion 

As  expected  the  Hybrid  net  trained  faster  than  the  back  propagation  network, 
the  net  sbe  sras  greater  than  expected. 

which  is  most  probable  due  to  inefficiencies  in  the  Kohonen  layer.  By  reducing 
the  Kohonen  layer  to  partition  fewer  decision  regions,  the  back  propagation  layer 
can  be  reduced  as  well. 


Samples 

ISSSI9 

First 

Second 

Training 

Right 

^ISt 

Right 

Guess 

100 

IHHd 

12 

24 

12 

81.0 

66.0 

12 

92.0 

1  652 

■o 

WSl 

24 

91.0 

KEI 

Table  11.  Hybrid  Propagatioo:  Kohonen  Nodes  vs  Samples 


B.5  Targtt  Identification  Data 

The  target  identification  data  is  a  subset  of  the  previous  data.  Those  blobs 
identified  as  target  were  subjection  to  addition  analysis.  Eighteen  statistical  mo¬ 
ments  were  calculated  for  each  target.  The  targets  could  be  either  tanks,  trucks 
or  jeeps.  Using  an  *  -iroach  similar  to  that  used  for  the  target/non-target  data. 
Differing  net  sized  gave  the  results  shown  in  Table  12 


m 

Upper 

Training 

Right 

Guess 

■2] 

Guess 

Converging 

81.4 

No 

10 

79.2 

55.0 

No 

15 

15 

63.3 

81.4 

58.2 

KII 

No 

40 

63.2 

78.4 

52.5 

71.2 

No 

Table  12.  Back  Propagation:  Accuracy  vs  Net  Size 


Using  the  Hybrid  Propagation  Network  Table  13  shows  the  results. 


1  Netsize 

1  Kohonen 

Lower 

Upper 

Training 

Right 

Guess 

Guess 

Cycles  j 

■m 

10 

10 

1^1 

2.1 

25,000 

HI 

10 

21.6 

46.7 

25,000 

mSi 

55.2 

74.1 

Table  13.  Hybrid  Propagation:  Accuracy  vs  Net  Size 


Appendix  C.  Computer  Source  Code 


This  section  include  the  computer  softw&re  for  several  programs. 


init.net(initialize.c) 


yssra  J()  %  1  0: 
x=((floiit)>7l00.0-0.5): 
rtitnru  x; 

1 


>/..55  Sor  to  I98S 


Ptff  3  of  imtUoli;r.c 


(inakeinput.c) 


* 

•  DATE  i  on  19SS 

•  yERSlOS:  / 

• 

•  ^AME:  Makttnfai.e 

•  MODVLE  SVMBE3:  I.4 

•  DE:  CRlPTlOy  Pnt^it  fcr  fiU  mpul  ■>/  «  daU  tet. 

•  AlOORfThM:  /V«»f 

•  PASSED  yARlABLES:  .Vowt  10 

•  RETCRyS.  Sent 

•  OLOBAL  VARIABLES  VSED:  Napt 

•  GLOBAL  VARIABLES  CHASCED:  Namt 

•  FILES  READ:  lapat  data 

•  FILES  WRITTEN:  Name 

•  HARDWARE  INPVT:  Nam* 

•  HARDWARE  OUTPUT:  Nenr 

•  MODULES  CALLED:  matmlj 

•  CALLING  MODULES:  Name 

•  2(1 

•  AUTHOR:  Grtparp  L.  Tarr 

*  HISTORY: 

» 

#iucln«l«  "daflaltioBa.k” 

#iurhi<l«  ”a«t.b" 

^jHucInde  <«(<1io.h> 

#iucln<le  <nMth.b>  » 

hit  «xani|>lM«  : 
hit  wetor; 
hit  exMi).t««t; 
hit  claaws; 

#d<«iw  TRI'R  1 
#d«iiii«  FALSE  0 

#4h(flllff  ni<tx«xiuiiplara  MN)  4« 

#dcfliie  niaxMwii.U-al  200 
#dcfiita  maxvvTtor  40 

fliHit  d«ta{nuucrxMnpl<«n-fnMxvx«in.t<4t][nHu(wrtor]: 
flout  diil«.lni(ni«X(rxMiii.Ufat][iiiaxv«(tor]; 

hit  rlwM{tiiiu((>xaniplaral: 
hit  claaa.l^atlmaxrxaiii.tr**': 

float  add.noiapsO.O:  M 

float  data.stata(iiiax(*xaiii|>lan][inaxvrctor]; 

oxteni  float  randomO: 


.'/;.5.5  Sar  '0  I9SS 


Paft  I  af  maknmpat.f 


rJuir  fil«n«iii<‘[30]; 

atittic  citer  sinwave(7]  =  { 
iut  do.aiii  =  TRt'E; 


iiiit.data(  makeioput.c) 


FILE  *fot; 

init  (lats( ) 

{ 

iut  ij,x: 
float  y.x; 

FILE  •fp.’ff: 
riiar  target['t]; 

printfCNaOata  FllMMa«:\a  **): 
aranfl*^'*.rilmani«); 

ioopi(T){if  (siuwav^i]  !s  fll<tiainr(i|)  (do.aia  s  FALSE;break;  )) 

ifldo.attt  ss  TRUE)  (init.ainO; 

raturu:) 

rirt=ropm<  "data.a«ata" ); 

fpsropru(  nivaaiiie,''r" ); 

iflfp=sNULL){  prinin"\a****  Fila  not  Fovad  ••••\a'’):fxil(0):} 
prtntR'VaOata  lastalladVa"); 

&caiiflfp,"M  Xd  Xd  Xd".&<Fxaniplsra,&«xatn.teBt,j:vertor.A'cla8wa); 
output  =  riaaam: 
input  =  v»<tor; 

printn"Xd  Xd  Xd  Xd\a".oxaiiiplan.«xain.t«wt.input.oulput); 
for  (is0'.i<rxainplara+rxain.trat;i4-f ){ 
liiraiifirp.'^  ".i:x); 
fi»r  (jsK);j<viwtor;j++){ 

r8ranrirp."Xs  ".i:data(i)Ij]); 


fiKanr(fp.'^''.largrt); 


CO 


TO 


PO 


1(10 


ii;.5.5  Sop  W  im 


Poft  t  of  moketnpoi.r 


o 


■ 

► 


> 


I 

I 


iiiit.d«t«-MAKE.INPUT(inakeinput.c) 

if  (t«rgH[Oj  ss  >T')  rlMi(i]«0; 
if  (tMget(l]  ss  *A')  rlaM(ijal; 
if  (t«rgp«(oj  ss  '.s’)  rla«{ijs2; 

■  III 

if  (t«rgfl[0]  ss  *0')  rliMi{i]sO; 
if  (t«rget[oj  ss  '1')  rIaaafijsO; 
if  (l«rgK(o|  ss  '2')  rlMai'ijsl: 
if  (t«rset(oj  ss  'J')  rliMH{i]s2; 
if  (t«r^H[oi  ss  *4  )  rl«^j(i]s2; 
if  (t«rgK(0}  ss  >i')  cla«i(i]s4: 
if  (t«rg«t(0]  ss  'S')  ciaMfijsS: 

} 


iw 

rdoa^fp); 

for  (isO;i<rxMi(.t««t;i-f-f)( 
for  (j=0j<ve«tory++){ 

<l»ia.t«M(i]D]s  «l»U(i4ex«nplan]ti];) 
rla«.U'at{i]srlMi(i4«x«iiipl«r^: } 

..  ISO 

l*r*UuMr 

*/ 

} 

iut  wuiipir.rtypr: 


MAKEJNPrT(x.doft.fiMidx) 

floHt  xQ; 
floMt  Jo^Q; 
iut  rMKfx; 

(  iut  i; 

Wdo.siii  ss  TRUE)  (MAKE.SIN(x.(kift);roturu;) 

iftrninJx  <  0  ){ 

rt}'|>r  s  nwdO  %  oalput  ; 

oMiipIr  s  nuKl()  %  ex«nip)«n;} 

elM  {samplr  s  nwHx: 

ctypi*  s  rlaa^suiiplr];) 


MAKE.INPUT 


Hn 


I  to 


wbiM  rliwo[iMMup><«)  )s  rtypc) 
Miiipirs  riuHiO  %  rxMiiplara; 


C 


Puft  i  of  matrmpo/.c 


'■Jr-*' 


il.  JS  .Voo  JO  l9Sa 


MAKEJNPUT-<lata.stats.fiinction(ii]akeinput.c) 


/•  fTtmtf(’%i  %d  :  */ 

«  * 

fur<isO;i<v«ctor:i+-f  )x[i]3  daU(Mnipl<*][Q; 
for(i=0:i<veftof.i++)  x(i]  +a  xp)  *  ruidoitiO*  Mld.iioise; 
for<  isO:i<o<itput  ) 

iflci  mftx  mptf )aai )  doApjsTRl'E: 
elM  doftp]a:FALSE: 

} 

A  K  E.TEST  ( x.doft .  irliirli ) 
flntit  xQ.donQ; 
iut  wliirh: 

{  iut  i: 

tamptf  =  which; 

ctype  a  ciMi.tcstiMnipIc]; 

fbr(  isO:i<  v«ct<)r.i-f4  )xp]adsU.i«at(whkh]p]; 

for(  isO:i<oMtpui;i4'+ ) 

iftcla«.U^!which]sai)  do(tp]aTRI<E: 

«Im  doftp]=FALSE; 

} 

f|l<icifliie  wIlexMn  («xanipUni4«xKnv.t«si) 

nirsn[Mr][in«xcxatnplarM-inaxexMi^l««t]. 

8l  dd«v(8ix«]{ni«xrxwiipl«i»4-uiaxcx«m.t<«t]; 

dat».»tata.runciioa( ) 

(  bit  nn.ij.kJMiall.right.wTonf.totrigbt; 
bit  riwM.MCuniBtMic]; 
fkMt  dMl{niKX<'XMnpl<in],l«mp; 

FILE  *fnig; 

tcxtport(0,1000.0.700); 

fiiig  w  ropcnClMrlaigh"."*"): 

JI;SS  /for  to  1908 


170 

MAKE.TEST 


tao 


11*0 


(lata.stats.fiiurMou 

VKi 


}t0 


P9$t  4  <*/  m0iifinp9tc 


dataLSt«t«.function(inakeinput.c) 


prinl((**ril«:  Xa  \a".lil«iiMne): 
fpriiitil(rnig,'*ftl«:  Xa  Xa^.Rlmame); 
print(i("lnb«r  ot  Saick  CXaaa;  VaXa*): 

/•  CuletUtt  of  (ock  orclor  •/ 

loo|ti(oiitpul)  claa*.acrunia(i]sO: 
loo|>i<allexaiii)  rlaM.accuniB(claM(i|]-M-; 
loopi<outpui){  printm-Claaa  Xd:  Xd  ".i.claaa.actuinati])-. 

rprint(1(ruig,"Claaa  Xd:  Xd  ".i.daM.arrnnMO]):} 
liiH>;fpriiitn  fnig,‘'\a‘' )' 


loopj(  input) 


(k>opk(output ) 

mMii{k]{j]  s  O.G; 
loopi(alk‘xani){ 

iiiraii(rlaai(i]][j]  =  ineM((laai{i]]()]-Hlata(i]G];) 
loopk(outpnt)( 

mMn(k](j]sinm(k](j]/claat.Brrnma{k};  ) 


#ir  FALSE 

/•  ffow  cotrnUU  tioniori  Jmttion  •/ 

loopj(input)(loopk(output)  sUldrv[k][i]  =  0.0: 
loopi<ail<n(wn)( 

prinif("Xd  Xd  Xd  \»"  j.k,i); 
•td<K>v(fla«(i]](i]  -fa 


inemn(rlaat(i]][)]-<iata(i]g]* 

nir«n(clana(i]][i|-dataii]y];) 

loopk(o«itpiit)  {*tdtWv(k](j|a8tdd««v{k)ll)/fliwa.afnin»(k]; 


/V#i7  JO  toss 


Pm/t  5  of  makimfot-r 


u..::  u' .V.  u  ‘  l::  • ; ;  4.f 


dat«.st«ts.functiou(iiiakeiuput.c) 


)  line: 

#«ntUf 

/*  Vttmf  ikr  mt^m  for  trrk  rloiu.  eoleuUit  duiomrr  from 
rock  cUm  •/ 

right  s  0; 

loopi(aUexMii|{Ioof>k(oulput)  dist(k]  a  0.0; 

loopj(  input) 

{  loo|'k(OMtput){  ttrnip  a  «lntii^][j]— niFaii[k][)]; 
dMt(k]  +a  Ump  *  mnp  ;)} 

stnnIlaO  ; 
t«flipadist(0]: 

/•  pn«#/rSmp.JFd  -.ij;  •/ 
loopk(output)  {  /•  prtnifr%S.^f  '.dutfk]);  */ 
if  (temp  >  dMt{k]) 

(  mud!  a  k; 
teiMpadiit(k];} 

} 

if  (rlna^MiMlI]  as  rlanaji])  rigbt-f't': 

/•  primlff  ”  C1o*$  fH  Srorett  Clo$$:%4  '.clo»$(iJ,imoll};Uat: 

7 

)  prints  "latiantu  ulng  elau*  MunaXa"); 

rprintnfiiig.'^tlantu  uniag  MUMNa”); 

printfl"  Xd  ri(ht/X4  \B''.right.allrx»ni); 

fpriiitflfnig,"  Xd  rlght/Xd  \a''.right,allfxmu); 

for  (nns  l;nn<IO;nn-t"t-)( 

printfCVa  Xd  nuttMt  ■•igkbors  ".nii); 

rprintf(fnig.''\a  Xd  auurMt  luighburu  ".iin); 

totright  a  0; 

loo|ii(exMii|>l)trs) 

{loopk(examplan)  di)it.(k]  a  0.0; 


STO 


MO 


wm 


300 


-I 


Ij 


310 


! 


.Vor  10  I9S0 


Pofc  6  of  moktnpul.c 


dat«.«taU.fttnction-iioniialiM.data(iiiaJceinput.c) 


/• 


V 


loo|tj(iiiput) 

{  loopk(exiiniplMii)(  Ump  —  <lftU(i][i]-<i»upi]|i]; 

«liM(k]  -fs  Uiiip  *  Iniip  ;} 

) 

niMlIsO  ; 
t<mpsdial(0]: 

fnutf(~Samftt:%i  nrt^hon 

S*mplt:%d  tlass.'ffd  ntifUcn 

riglit  a  0;w’.t>ag  s  0: 
for<ls  0:l<uii:l-M-)( 

•mallsO  ; 


trtiips<liiU(Oj; 

loopk(<‘XMiiplan)  { 

if  ((tr^ip  >  <liat(k|)  k  (i  !a  k)) 

(  MMlI  a  k; 
u^ipadiaifk): 

) 

1 

dMl[anwUI]3|U00; 

/•  prtmtf(  '%U 

fpnutfffnif.  '%Jd 

V 

if  (rlaM(si»«i)]  as  rlMRji])  riglil-t>>f; 

obe  wroug+-h; 

) 

iflrigbO wrong)  totright-f-f: 

} 

printf("T«t«l  right  Xd  X3.2f".torright,(lkMit)(otriglit/(8o«t)pxampbtr«); 


rjiriniflirnig.Total  right  Xd  X3.3f  X" 

.toiriglu ,( flont  iioiriglit  /( flont  )FXMnp)M8*  100 . ); 


} 

frloM^rnig); 

rxiMU): 

} 

(  ini  t.jsO; 
flcMt  n\n%: 


330 


330 


Mil 


3i0 


300 


uormalizc.dcxta 


3T0 


A'or  W  l9St 


P*ff  7  of  moknmpnl.r 


iiorinaliM.data-init.sin(  inakeinput.c) 


nwg  s  0.0; 

ioat>i(inpal)  nwg  4s  <laU(i][i]  *  <l*U(j]{i): 
mag  s  ^rt(nwg): 

loopiiiniMit)  <l!U«(i][i]s  «lal»(ij(i]/nwg; 

)  prints  nteta  lozMliMdVji"): 

) 

#dnflM  PI  3.Mt5 

MAKE.SIN(x.doft) 
float  xj; 
float  doftQ: 

(  float  y.s.ai>*<il.|ilMM: 
kaig  uo«; 

iut  ij,k.«bicfa.wbu'h2.wliicli3,«hkh4; 
iinui<l(tiinr<l-iioa)  %  37  ); 
wbirb  s  randO  input: 
phaar  =  raodoinO  *  PI: 

kMipi(inpul)(  x(i]s  raa(((float)i*PI*(float)whirh|/inp«it4  pbaip):) 
loo(>i(outpul)  d^(i]  3  0.U:(ioft(»hkh]3l.0; 

pbaar  s  ran<loni<)  *  PI; 
irbkiil  3  ran«l()  %  input; 

loopi(input)  x(ij  s  x(i]4  raa(((float)i*PI*(float)«bieb2)/input4pbaaF); 
doA(wbicb2)  s  1.0; 

phax^  3  raadoml)  *  l*l'< 
wbkh3  3  raml()  %  input: 

hio|>i( input)  x(i]  -  x(i]  4  •iii(((flaat)i*PI*(fluat)whirb3)/iuput4pbaaF); 
<loft(«hkh3]  3  1.0; 

) 

iiiit.Mu( ) 

{ 

print(|"l«w  aaay  lapat  and  Output  lodaa  ?"); 

■raiiQ  A'output ); 
input  3  output; 

F.xaiiiplan  s  0; 
rxaiii  tnt  3  0; 

} 


iV»i>  JO  mo 


MAKE.SIN 


3(H) 


40f) 


iiiit.siu 


410 


Pn§t  S  of  moknopot.c 


FEED.FORWARD(f«edferward.c) 


DATE:  i9  Sfpl  I9S8 
^  ^ERS^O^’:  t 

NAME:  FttifonMT* 

MODULE  NUMBER:  t.S 

DESCRIPTION:  Ptoftftitt  4ai»  from  Ikt  i»pml  to  ikt  oulfmi 

ALOORITHM:  Sioniant  ftr4forward  ralrt  in 

PASSED  variables.  Nomt 
RETURNS:  .Von* 

GLOBAL  VARIABLES  USED:  Noat 
GLOBAL  variables  CHANGED:  Noat 
FILES  READ:  Noat 
FILES  W  RITTEN:  Noat 
HARDWARE  INPUT:  Noat 
HARDWARE  OUTPUT:  Noat 
MODULES  CALLED:  Noat 

CALLING  MODULES:  Noat  m 

AUTHOR:  Cnforf  L.  Tarr 
HISTORY: 


#ittchMlfc  "dnflaitiaM.k" 

#iuchi<le  "■•t.k" 

3U 

#inchi(le  <in«th.h> 

«xtfini  ntroct  nraraljiot  imi: 
fliMt  rnlryi): 

FEED.FORW’ARDO  FEED.FOR\\’ARD 

{  lilt  ij: 

III 

loopi(inpnt)  if  (noi.ii(p.miwk[i]  ss  0.0)  not.inp(i]s0.0; 
loopi(hi<lo.onr) 

{nrt.yl(i]s  *aky(ii«t.inp.iHt.w0.n<>t.t0.fci.iiipiit); 

) 

loop«(hido.ono)  if  (iMft.yl.inMfc(i]  ss  0.0)  art.yt(t]s0.0; 
loopiiliido.lwo) 

{tiri.y2(i]s  ralryInf’t.yl.iKH.irl.niH.il.Ai.liHlo.oiio);)  no 

loopi(linlo.tiro)  if  (iiPt..v2.imMk(i]  ss  0.0)  npi.y2(i]=0.0; 


/ 


Noo  M  1938 


Paft  I  of  fttdforwari.r 


FEED.FORWARD-flndnode(  feedforward.c) 

laopi(  output) 

(n^.outp(i]sral<'y(nrt.y2.iiH.w2.iKi.t2Jci.hMl<>.lwo):) 
ioopg<  hiiie.one.iiiput ) 

{  iflmft.y2.masl([i]=sl.O)  iiK.iiwO(i][j]  =  n«l.«rO(i][i)  •  uH.iup[i];) 

■Ml 

loo|>g(  hiJe.two.  hiUe.oM ) 

I  iKnM.}rI.inaak(i}ssl.O)  =  iin.trt[i][i]  *  BFt.yl[|];) 

loopg(outpul  .hidr.two  | 

{  iltnrt.inp.ntMkyjaal.O)  iivt.aw2(i](j]  *  ■iet.w2ti)(j)  •  nH.y2(i];) 

)  TO 

flout  Rxy(); 

flout  .'airy(x.w,tliot«.in«lvx.lo«rr) 
flout  x(],wQ{iiix^],(hrt«{}; 
bat  *in«i«x; 
bat  kw; 

I  bat  i.k.iiumtMw: 
flout  y; 
y  *  0.0; 

nunilMPf  a  *in<lex; 

loopi(lowor){  y  =  y  4-  xpl  •  w(uuiii(>or]p];}  m 

y  a  y  -  thvt«(liuillb<^; 
mtttraa  fixy(y.2.0);) 

flout  fixyCy-.liardnuH) 
flout  y.buHiifw; 

ion 

r«tiana(  1.0/(l.0  +  (flaut)oxp<-(doaablu)(liiurdn«si  •  y)))); 

nn.lno.l«ix».y#)  flUtluotU’ 

bat  xs.yo; 


Jl.se  iVoo  }0  J9S8 


Ptft  i  of  ftrilfortp»r8.c 


&iKlno<le(feedforward.c) 


(iat  i.iayvr.tli^.nodr.tmnp: 

Uyw  a  (y«/«|irilig): 
th^.iKxW  X  (iutMxs  /  weight.*); 
t#fiip  X  ( (02-t/xFight.s); 
clww.sf  w.i( ); 

•witdMiay'Y)  { 

CUM  0:  liiF.iiotir  -X  (Uwip— m|Mtt|/2: 

if(art.iapjitMk(tbF.iio>l^  xs  0.0) 

{  iirt.iBp.iiia*k(th».iio<lr]  x  1.0: 

loopi(hkla.uMr|  nrt.tOfi]  -fx  iM4.a«r<lfi](llir.no«lr]:  } 
mIm 

{  ■i«t.inpjnMik(tbe.iiod^  x  0.0: 

loapi<hi<)r.onF)  nM.tOfi]  -x  nrt.axOOKtliP.BOtie]:  ) 

biaxfc: 

CUM  I:  tHe.noda  -x  (lmip-hk)«.oiM>)/3: 

xx  0.0) 

(wt.)-|.tiMNk(thajKKie|  X  1.0: 
loopHhMr.ixo)  nrt.llp]  -fx  ort.axlpjltbe.aotlrj;  } 
OIM 

{  iwl.)rt.inwk(lhp.iiO(lr|  x  0.0: 

toopilhirfp.txa)  art. tip]  -x  iiri.*w)(i](lbr.iK><ii^:  ) 

brmk: 

CUM)  2:  thr.node  -x  (i<^ip-hHk.txo)/3: 

i0|nrt.}'3.iiWMll(lh*.no«lr']  xx  0.0) 

(  iK-l.vS  nuwk(lhrjKxlr’)  x  1 .0: 

) 

ela« 


.Vac  W  I9SS 


iw 


ISO 


14* 


1(0 


Pm§t  J  of  fttiforwri.c 


fl  nd  iiode(  feedforward  .c ) 


{  tHH.y2.mMk{th«.no<ie]  =  0.0: 

break: 

) 


} 


W  iVor  10  198S 


JIM 


P»f(  1  of  frrIfetirirO.c 


TIlAIN.NET(traiunet.c) 


• 

•  DATE:  JS  y«/j  1988 

•  VERSION:  I 

• 

•  NAME:  Tninitrl 

•  MODVLE  SVMBER:  1.6 

•  DESCRIPTION:  Vainf  onlfat.  aJjuals  wighla  rrdaer  rmr. 

•  .ALdORlTHM:  Wrriaa  MaUiUftT  Ptrctpiron  BarkprofMfalioB. 

•  P.\SSED  VARIABLES:  Nont 

•  RETVRNS:  Nont 

•  OLOBAL  VARIABLES  USED:  HVij*/  Vtclora 

•  CLOBAL  VARIABLES  CHANGED:  Uagkl  Vtclora 

•  FILES  READ.  Nont 

•  FILES  WRITTEN  Nont 

•  HARDWARE  INPUT:  Nont 

•  HARDWARE  OUTPUT:  Nont 

•  modules  CALLED:  Nont 

•  CALLING  MODULES:  Mnin  Loop 
0 

•  AUTHOR:  Grtgorg  L.  Tarr 

•  HISTORY: 


#dciii>«  THFTTA  TRI'E 

#iuchule 

#iiiclnvl« 

«xt«ru  stnict  nrural.iivt  n«t: 

n•*ta  0.3 
flout  (Mx(); 
float  <My(); 

#didiua  nK>iiM>iii  t.i,.  J.7 
TRAIN.NETO 

{iut  ij.k.ii; 

float  ii«.’l3[«ix<']  .(I<*t2[*ixt']  1  ["■*<*] : 

/•  onipnl  */ 

loo|)j(o<ilpul)  {  JHSQ]  =  <Wy(iwl.outp[j),net,doftg]); 
luo4>i(lii<l>‘.two){ 

m'l.w2.nK(m[i](i)  =  (n^fn  *  (Irl3(j)  •  nrt.y2(i)+ 
.\»r  10  1988 


?n 


'/ 


an 


4a 


TRAIN.NET 


5n 


Puff  I  nf 


fliidnode(f««dforward.c) 


{  iiot.y2.ni««k[tlw,iKxie|  =  0.0; 

} 

break: 

) 


1 


HM  Sor  iO  19SS 


tmi 


Pnft  4  of  fttifotruTi.t 


Q 


iitonimiuiu  *  nH.«2.DK>iii^}r*]) 

*  ii«t.y2jitask{i]; 

ii«i.w2(|][i]  +=  i>rt.w2,»iK>m[j)(i): 


TRAIN,NET(traiiinet.c) 


)) 


/*  Stcomd  Biddtm  */ 
loopj(hid^.tiro) 

{  d(‘l2(j]s  <lvlx(ii«t.y2Q],<ki3.iict.«v2j.output); 

/•  mtUlQ]  ntU  •  ;•/ 

loo|ti(hict«_one>  ( 

n«t.wl.nK>in(ij(ii  »  (nrt»  *  <l^l2[j]  *  iipt.yl(i] 
-f  momratiiiii  *  o«*.w)jix>m[i][il) 

*  tiet.yl.iiMMkfi]: 

nrt.wl[j){i)  +=  iw^.wl.momlilpl: 


}} 


/•  First  BMrn  •/ 

/oopjCliide.oue)  {  <MJ{j]3delx(iirt.yl{(].(i<rt2jKt.wl JJiide.two); 

I*  mtUOfii  +s  •(<«  •  itUliJ  :  •/ 

kMpi(  input)  { 

nei.frO.niommi}  = 

(  iMriu  *  *  Rpt.iupfi]  +  iiH>meutuin  *  iiet.wOjiioiu[i](i]) 

*  M^.inp.mank^): 
n«4.w0(jjp)  +*  wt.wO.inonipJpJ: 

)  } 

} 

floMt  d«rly(y.(Jofl) 
flkmt  y  ; 
flunt  (toft; 

(  floMt  (WIsO.O;; 

(M  s  y*(l.O-y)*(<loft-y): 

mtnr^i  (W: 

) 


*0 


m 


m 


90 


lUD 
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Poft  3  of  Imnmrt.c 


TRAIN.NET(trainnet.c) 


SoMt  ilHx(x,del.n.n.iipp«*r) 

flout  x.drtD.wO[siae]'- 

iut  n.npper; 

{  float  (MU^um; 
iut  ij; 
mini  =  0.0: 

k>opi<ttpp«r)  iuin  =  iuni  +  del(i)  *  wplM; 
mini  =  x*(1.0-x)  *  *nni: 
retuni  stun: 


•1 


iJ:S7  ,V*r  30  1993 


P«ft  1  »J  lt*i»nii.c 


SHOW(show.c) 


• 

*  DATE:  1  Octoitr  I9S8 

*  \  ERS10S:  S.O 

* 

•  SAME:  Skow.c 

•  MODULE  SVMBER:  11 

•  DESCRtPTlOS:  Uatd  to  8i»fUf  $mttmol  roiuta  im  Ikr  Urtfori 

•  ALC.ORITBM:  Soot 

•  PASSED  VARfABLES:  Soot 

•  RETVRSS;  Soot 

•  GLOBAL  VARIABLES  USED:  Soot 

•  GLOBAL  VARIABLES  CRASGED:  Soot 

•  FILES  READ:  Somt 

•  FILES  WRITTES:  Soot 

•  HARDWARE  ISPUT:  Soot 

•  HARDWARE  OUTPUT:  Soot 

•  MODULES  CALLED:  Soot 

•  CALLISG  MODULES:  Soot 

• 

•  AUTHOR:  Grtfort  L.  Torr 

•  HISTORY: 

• 

#tBcliido 

#iuchKl«  <M<iio.b> 

#iu«'hulr  “datiaitloM.k" 

#«k4liw  TIU'E  1 
#iMlu«  FALSE  0 
extern  int  dispUy.iio.MM: 

extern  rtrnet  nrarsi.iirt  n«t; 
extern  float  arcnrecy; 

extern  nit  ro<mi,rigiit.gn««i,good.t««t,<ieriMona; 

hit  10  s  0; 
hit  II  s  0; 
hit  llw  =  6: 
hit  12  =  0: 
hit  13  s  0; 
hit  14  s  1; 

SIIOWO 

I  ifllfl)  •howinptiMnei.inp)  ; 

iflll)  slmwiKxMnrt.yl.hide.one): 
ifliltr)  Khowwi^ghtidiHH. wO.iiri. to. l.likle.one .input): 
ifll2)  slioeno<Hnei.y2.hkle.t'ro); 
ill  14)  shoeoulpuMnet.outp.net.don.): 

» 


yt 


90 


40 


SHOW 


s« 
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Poft  I  of  akotr.f 


SHOW-^liownod«(show.c) 


•bowrinpuMx) 
flout  xQ: 

{  ittt  j.i; 

imntrcXsltllayar  Parcaptros  RedalNA**): 
loopi(i]iput)  prinlfC  X(Xd)  '*.i);tine; 
loopi(iiiput)  priaifl"Xl.31  ‘'jiri});fina:Une: 

} 

siio«oiUpul(  y  .doft ) 
float  yQ: 
float  iloOQ; 

{  iut  ij: 
float  arror; 
printfi(”Oat 

loQpi<oatpiii)  priiitr("X3.3f  ‘',y(i]);lina: 
prin»/i("OafT.'"); 

loapi(otitpui)  priiUl("t3.2t  ".dodti]);  linr; 

print(\"Caaat:M  \atigkt:Xa.3<  6«aaa:X3.2f\a'',ctiunt, 

(float  )right/(  float  Hixplay*  100.0. 
(float  Igood/ifloatHlMplay*  100.0): 

aido.iMn  !a  TRl’K) 

loQtti(ottlpHt)  printffClaaa  Xd  X2.3f  Na".!, 

(float  )nrt.claa«.tMt{i]/ 

(float  )ii<'i.(lMii,couul[i]*  100.0); 

linr; 

arrnrary  s  (flaat)d«ciaiom/(float)<lixplay/(float)o«tput  *  100.; 

prini((”Da«iaiaM:  X3.3f  VB^.IfloatltlrrMoiia/ 

(float  )()ispla}y( float )output  *  100.0): 

clrM.traiO; 

1 

xhoano0r(y.ii) 


il.SS  Nor  30  JOaS 


sUowiuput 


sliotivtitpnt 

TO 


00 


too 
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•hoiniode-showireighta(show.c) 

Soat  yQ: 
iut  n; 

{  iut  ij;  HO 

float  nwx.niin: 
line; 

printf  ("lod«  Val«M:*');line; 
loof>i(n)  prillin'^- 4f  **.y(i]):liiie: 

}  i» 

•iiow-.<eigiila(  «r  .thela.layer. upper  .lower )  sllOWWeigLtS 

float  w(](fiife],tlieta(]; 
iut  layer.upper.iower; 

{  iut  j.i: 

float  inaxjnin:  lae 

line; 

loopi(upper)  print(("Vl(Xd,Xd)  **.layer.i):liiie; 
loopi(lowrr)  { 

loopj(upper)  {  if  (w(j)p)  <  0.0) 

priiiinc^l.Sf  ".wplpj); 

aba  140 

printfl"  Xl.W  ".wp)p));} 

line;) 

priiitl)  "NnAatas:  "); 

loopi(upper){  printfl  "  XS.Of  ".theUp]);} 

line; 

}  IMI 
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(nienu.c) 


« 

•  DATE:  1  Orto*fP  im 

•  VERSlOtf:  1.0 

• 

•  NAME:  U€mm.e 

•  MODULE  NUMBER:  1.8 

•  DESCRIPTION:  Prondt$  tnUrtfiipt  mtnm  fmuciiow$ 

•  ALGORITHM:  Nomt  i" 

•  P.iSSED  VARIABLES:  Nont 

•  RETURNS:  Nome 

•  GLOBAL  VARIABLES  USED:  Nome 

•  GLOBAL  VARIABLES  CHANGED:  Nome 

•  FILES  READ:  Nome 

•  FILES  WRITTEN:  Nome 

•  hardware  INPUT:  Nome 

•  HARDWARE  OUTPUT:  Nome 

•  MODULES  CALLED:  Nome 

•  CALLING  MODULES:  Nome  » 

• 

•  AUTHOR:  Grefon  L.  Toer 

•  HISTORY: 

• 

„ee„„„„e.,.,e,ee„„.,„„e,;;*oo;oooooo»o,oo*.oo,o,o,ooo»oo,„»»f 

«xt*«a  tut  dii.rlaaijMniple.exMiv.tMt; 
oacteaeu  flout  «d<ljia«e; 

«rt«m  int  aciivjdKMv.irnKhfai.dMpIsy; 
ext«ni  float  Ihreshold.add.noMe: 

ao 

#iurhi<lo  <n(p«iil.h> 

#iurJiHi«  <ttdio.h> 

#iBebMl«  "daflnltlOM.k" 

«xt«ni  iat 

mrtoni  stnict  nmral.avi  nrt; 
wrtoni  fait  <M<inl.ri$bt4ood4ti«aa,teit; 

TRUE  1 
#<kifliie  FALSE  0 

hit  loraid )  40 

{  chiv 

hit  iiod«i»iiiib«r.wt«vlorsTRUE: 


priiitr  ("  R«m  V*"): 

printr  ("  1)  Xaitlalis*  Syat«a''):liiw: 
priotr  C  3)  Sm  V«ickt«''):iin«>: 

ininlf  ("  3)  kaad  Voigkt*  "IdiiH*;  m 

priiilf  4)  Toggl*  tct/V«l(k«a'').liii«; 

liriiiif  C*  S)  Add  ■«la«''):linp: 

iwintr  ("  •)  Olaplay  Xat«rT«la");liap: 
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iiew.net(iiMnu.c) 


printf  ("  7)  Teggl*  Irror*  '');linv‘, 
printf  ("  $)  IM  lat  Topology** 
printf  (**  9)  Data  Statisties'*);Une; 
printf  (**  a)  Sot  Stop**):line: 
printf  ("  o)  EXIT  \a"): 
printf  (**  SKLBCnai:  7  "); 

arWt  s  grtcliar(); 

■witdifaolort)  { 

caM  *0*; 

goxitO; 

syat«ii(  *‘geloar** ): 

riuKO); 

hmak; 

CUM  *  1  * ;  init.dataf ); 

:braafc; 

caa«  *2*;  writ<‘.nrighto  (); 

;btmik: 

caoa  *3*:  priutfl *'\alat«r  flloniao:  \a  7  **); 
sranfi(  *'Xa''  .Hlonainr ): 
read.irviKhta  (Atmatnr); 
printff**'^  Xa  Iaatallad\a\a*‘.fUenanw); 
braak: 

caM  *4*;  artiv  =  TRUE; 

btaak; 


caoa  *S*:  printR’'floo  aach  aoiso:  7*); 
sranR  *'X1*' .kadd.noiK): 
braak; 

caaa  *6*:  priniit*'\aCoaat  botaooa  Scroon  Opdato  7\a”); 
•caaf(  *'X4'’  .Ifahow.wrighta); 
priiitR**\aCoaBt  botnooa  tooto  TVa**): 
acanff  *'X4''  .1:  diaplay ); 

hiwak: 

caa«  '7*:  inistakea  =  TRUE; 
braak; 

caaa  *9’:  ayatetnCeloar**); 
init.nrtf ); 

braak; 

caae  *9’:  init.data(); 

data.ataU.fnnrtkwf ); 
braak; 


•ignslf  SIC!  I  NT  .mrau ) ; } 
nrw.tirt( ){ 

print  f(  **100  lotoork  Topology  Xa**): 
printf!  "Inabor  la  Firat  LayorXa**): 
araiiR  **Xd**  .1;  hidr.onr ); 
priutn”laabor  la  Soooad  Lay«r\a"i; 


a 


40 


aJ 


rJ 

‘  .  a 

□ 

J 


li 

j 


lOfI 

uew.uot 


I 
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J 


new.iie 


•cauH  "td"  ,4:  hiUc.two ); 

): 

rrMl.wrighU  (filriuMiif) 
cliMT  fit<rnanip{]: 

{  FILE  •fp; 

BuMt  x: 
ittt  ij-k: 
fp  s  fopm 
if  (rpssNl'LL) 

{priiilf("\a  Srror***  \a");r«tiiru:} 

(iK»ut  (fp.*^  Xd  Xd  Xd''.4:oiitpm.4:bi<t4f,l«ro.4;liitl«.oiiF.4;iiipnt): 

kiopiji  output  .hulr.two)  { 
brauli  rp.m*  .4:x); 
iirt.w2(il[jl=x;} 
loopij(hiHr.tiro.hi<ir.on«){ 
liKaanrp.‘*X2**.4;x); 
iwt.wl(il[jlssx;) 
loopij(  lii<h>.oiM>,iiip)it )  { 
bc«irirp.”Xf*'.4:x); 
ii<'t.wU(i]li]sx:) 

loopi(  output)  (fiiranf(fp."Xf".iiM.t2-i>i):| 
loopi(hide.taro)  (larauflfp.'*Xf‘‘.u«t.ti+i);) 
loo|ii(hide.oii«)  {braii(i(fp,”Xf"ji«t.t(H-i);j 
faranit  fp,*^**  ,&rount ); 

loo|>i(  liid^.two)  ( bcanfi  fp.*^**  .nrt .y2.iniMlc+i ): ) 
ioopi(bitk>.onp)  ( fafanlll  fp.“X<"  jiet.y  Ijiiaak-f i):) 
lonpi(iiiput)  {i8ranf(rp."Xf",n<>t.iiip.tnaak-fi);} 
ffkiaHfp):) 

arritv.wvighta  () 

{  FILE  ’fp: 
hit  ij.k: 

cImt  fll«fnani«(20]; 
priiitfI''\a>Btar  filauM; 
acann'*Xa'',rilniaiiir); 
fp  =  fopMi  (fikuiame."*"); 

fprintf  (rp."Xd  Xd  Xd  Xd  \a",outpul,lii<Jv.turo,bi(ip.o»e,inpnt); 
loopij(ontput.hi«b*.two) 

fprilitf1(rp.''Xf  ".lirt.w2(i)[jj,ij); 

loo|»ij(lii«l«*.tw*).hi«l*.on«') 

fprintf; rp,"XJ  ".n«ft,wl[i](j),i.j); 
loot>g(  hid'.onr  .input ) 

fprintf(fp."Xf  ".n«^.wO{i][i],ij); 
t^iifontpnt)  {fprintrifp.'*X<  \a".nrt.t2(il):) 
loopifbnV.two)  <fprintf[fp."Xl  \a’*.wH.tl(i)):} 
lo>jpi(hi(i».oiw)  {fprintnrp."X2  \ji".n*t.tO(i)):j 
fprintflfp,'^  \a",rouiit); 

b>o|>i(bi»I».fwp)  {rprintf(fp."X2  \B’*.nct.y2.niaidt(i));) 
loopi(hiu**.on»)  jfpriiitltfp.'^  \B".nrt.yljnart(ij);) 
loopi(innut)  {fpriiitF|fp."Xf  \a''.nrt.inpjiiaak(i]);} 


i-write.w3ights(  iueuu.c) 


reatl.weigliLs 

110 


m 


lao 


write.weiglits 

ltd 
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frlosp(fp):priiitn'*\a  Haights  SteradXs**): 

} 


write.weights-write^oat(  iti«nu.c) 


writa.stu.weights  () 

{  KII.E  *fp; 
iut  ij.k: 

ciuir  fiienam<4?0|; 

fp  s  fopan  ("ataadard.aai","**'); 

fprinlf  (fp."td  Xd  M  Xd  \a-,output.lii<l*.two.lii»le.ooa.mp«t): 
loo|)ij(o)itput.bide.two) 

fpriiiir(ip."Xf  ",inH.w2(il(j] jj); 
looptj(  hi<lr.t«o.hi<ie.onv) 

rprint((rp.‘*Xf 

loopg(  hula.oa«.iap«t ) 

fprtiilf(f|>.'^  ".net.«rO(i)[j],i  j): 
loo|M(oulput)  Xa'aiat-tSli]):) 

loopi(hi<la.t«o)  {f|>rintf(fp.”1U  \ft".iMrt.tl(i)):) 
tootiHhiiKoiH*)  \n".uot.K){ij);j 

fprint(l(fp."Xd  \B*',ro«tnl); 

looi)i(hidr.t«o|  {rpriiil(?fp."tt  \*“.n^.y2.inaak{i)):) 
toopi<liid«>.oaa)  {fprinlf(fp,'*1U  \a".iirt.)rljMask[ij):) 
loo|H<iiipat)  (fprintflfp.Hu  \*".iirt.iiipjn«ak(ij):} 
f<'iose<fp):prtntn’'\a  Haights  StoradSA"): 

} 

wHta.iitring(x,y.l.tiila) 
iat  x.y.l: 
cluw  title(20]; 

{  rbiw  miiiilx^20]; 

I  s  (iiU)((fluiit)l/vHlao); 
rolorfti); 

rw»ll(x— 5.y— 5ji+l,y+l5); 

folor(4); 

linawkltM  1 ): 

racli(x—50’’“-‘* 

fnK>v2i(x,.v-2); 

rhsfiiMtilk*); 

} 

wrilr.no»t(  x,y.l,tilla.n..«.rok»f ) 

iut  x.y.l: 

cbiir  lillv(20]: 

flout  ft: 

iut  a.roior: 

(  rJuur  niimlM-r(20]: 

I  s  (utt)(( flout )l  /  vi<lao); 

Kpriiilfi  .  31"  .ft ); 

rolorf  a.roior); 


,Vor  M  im 


iw 

write.stcl.weiglits 


ITO 


!«» 


KTite.striug 

i?*© 


wrifc.float 


310 


P«ff  4  ©/  r 


writ«.float-writejnt(iiienu.c) 


n^tlKx— 4.jr-5,*+|,y+15); 

linrwidth(  1 1; 
nuov2i(x-2.>-): 
Hiantr(tiik'): 
rhantr(nunit>«r):  } 

wri»««Jnt(x.y.titl*.a) 

iat  x.y; 
cli*r  titl(>[2a]: 
iut  R: 

{  cImt  iiunil>rr(20}; 

spriiiia  nunib«^r."Xd"  .ft ); 
rinov2i(x.y): 
rhantr^titlr); 
rhanirf  number);  } 


write  jut 

33V 


33U 
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DISPLAY.NET(di«play.c) 


• 

•  DATE:  10  198$ 

•  VERSION: 

• 

•  NAME:  DISPLAY.NET 

•  MODULE  NUMBER:  1.9 

•  DESCRIPTION:  Duplet*  ike  Network  uiiuf  Grtpkirs  eoIU 

•  ALGORITHM:  Nome 

•  P.kSSED  VARIABLES:  Nome 

•  RETURNS:  Nome 

•  GLOBAL  variables  USED:  Net 

•  GLOBAL  variables  CHANGED:  Nome 

•  FILES  READ:  Nome 

•  FILES  WRITTEN:  Nome 

•  MODULES  CALLED:  Nome 

•  CALLING  MODULES:  Nome 
» 

•  .AUTHOR:  Grtforf  L.  Tore 

•  HISTORY: 


#in«hide  "aat.k” 

#faKlafla  "gl.k" 

MrtMV  int  couiit.rigbt.cood,t««t,<lo.Mn; 
«ct«ni  ftuMt  ii«ttral.n<H  iKt; 
bit  artir  s  FALSE; 

DISPLAY.NET() 


{ 

writ«jiiriiig(’IW.<5S2.50,''V«l««''); 

wril^.8triiig(5W).ttlV2,50,''Q««aa"); 

wrilr.«iritig((i80.tf52.50."ftifbt*'); 

wril«.string(520.700.130.''])aairad  Oatpat**): 

writ^jrtring(30.T30.350."Fa«dfexaard  Backprapagatloa  lataork"): 

): 

if  (!  activ)  { 

ploliiodf  (0,iK*t.in|>.iK*l.iii|i.inaak.nrt.y  ljiia8k.nvt.w0.in|>ut.liKl<>.oiH>): 
(tlotiHMV  (l.net.yl  .iiat.yl.fna«k.nvt.y2.nia)<k.n<H.wl.hi(lr.oiir.hulr.taro); 


n 


V 


30 


DISPLAY.NET 


40 
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DISPLAY.NET-plotnodet(dupl^}r.c} 


ploinodr  (2,net.y2  ,iiet.y2.uiMk,jn*t.o«itp_niiMk,iiH.w2,iii<{e.t«rQ,oiit|iut); 
writc.stringj  100,220.i20."¥«lgfct*‘*); 

} 

eb«  { 

plotiioOr  (0.art.inp.nM.inp.ina«k.u<t.yt.niMk.m>t.AwO,inpui,i«H<r.oiif>); 

plotnotir  .iirt.yljnMMk.u<ft.y2.mask.twl.airt.hid«.oa».lii(i«'.two): 

piotnode  (2.n«<.y2  .a«rl.y2.iiwak.iiet.oulp.miii4i.i))>t.«ir2.|iidejm>.oiitpiit); 

irrit<r.8(riug(100,220.120,"AetmtloB‘'):} 

pioiiio<M( S.net.ottip,  output  }; 

plotiio<M(  3.0, nri.doft. output ): 

tf(do.»in  «  F.\LSF,){ 

ptoiiiod«d(  3,  lOO.urt.doft.output ); 

ploiiiod«ti(3.200,iM.<}oft  .output);} 

*/ 

fb«rkjiio«»f< );  • 

) 

So«t  thmiioids  0.0; 
piotno<M<  x.iK>d««.ioim) 
iat  x; 

fluNt  nod*^; 
iut  '  r; 

{  iut  i^.k.y.x2.y2; 

ys«priiig*x-P30; 

*»(  102'l''Wrighi.8*low^)/2: 

loopiOoww)  { 

»»t,folor(  1 . 1 .0.01  .iiod<*(i]); 
l>(gj>lot(i*wt'iglil>+X.y.nrWglit.if/-(*.3); 

2i:X9  Sop  36  1998 


n, 


X)Iotuo<loi 


w 
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pIotnodei-plotnodex(di8piay.c) 


iftdo.iiin  =»  FAL.SE){ 

color.of(0.5.0.5.nod«f[i]); 

bigj>lot(  i*  wi‘ight,s+x+ 1  OO.y,  weight.*/ 1*3 ); 

fok>r.or(';.9,0.1.iK)dp{i]); 

l)ig_i  !o>(i*wMght  s+x-f-200.y,w«glit  8/4*3); 

) 

} 

} 

pk>(iK><M(x.y2,iio<i<‘, lower) 
bit  x.y2; 
float  nodeQ; 
bit  lower; 

{  bit  ij.k.x2.y; 
y=8|teiQg*x4'30; 
x=(  t024-weiglit^*lower)/2; 
loopj(  lower )( 
loopi(  lower)  { 

rolor.ofl 0.9,9. 1  .(float  )iiode(i] ); 

bigj>lot(  i*  weiglil.M-x4-y2, 

y+wetglil  8*2.weight  a/4*3); 

))) 

plotiKHlex(x.y2.iiO(le.luwer ) 
bit  x.y2; 
bit  llodeQ; 
mt  lower; 

{  bit  ij.lc,x2,y; 
ys8peiiig*x-f30: 
x=(  1024— weight  ji*lower)/2; 

■Jt:39  Sop  tO  198$ 


111) 


130 


plotuotlctl 


13U 


140 


plu^mxlex 


ISO 
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plotnoclex-plotnode(display.c) 


loopj(  lower) 
loopi(lowrr)  { 

8rt.color(  1 .0.0.0.(flo«t)iio<Jefi]): 

bigj>lot(i*w«ghtji+x+y2. 

y+w«glit.s*2.  weight jl/4*3); 

}} 


160 


#if  FALSE 

|>loliK><Je<x.uotle.array.lower.iip|>er)  plotUOtl? 

iut  x; 

float  nodeQ.arrayQfoixe]; 
iut  lower,u|>|M*r; 

(  iut  iJ.k.y,xljc2.y2: 
float  niax,iniii.lenip.t<'iiip2: 

y=6p<-iiig*x+30:  im 

xl=(  102-1— weighl.6‘lower)/2: 
x2=(  !02-l-w«^it.s*upper)/2; 
liiiewidlh(2): 

riiulniax(  array,  A'  nuix.ici>iin.upper, lower ); 
loopi(low«t)  { 

rolor.ofifO.O.0. 1  .nodeji]); 

Wg.;>Jot(  i*weiglit.s-f  X  i  ,y.weight.s/4*3): 

)»« 


loopj( upper)  {  tenip  =  array 

set  .rolor(Miax,niin, temp); 
drawii(  X I  +weight.s*i,y+weightji*3/-4. 
x2+w«ght.8*j,y+spciug); 

) 


} 

rolorbar(102't— 2-'i6.y+10.inax,niin);  mo 

} 

#<4t<lif 

plotnode(  x.iKxle.niaak.niask.up.array, lower. upper)  plotuodc 

iut  x; 

float  iio<le(],iiiaakQ.nMsk.op0.array[](site]; 
iut  lower.upper: 

{  iut  ij.k.y.xl.x2.y2: 
fl«iat  iiiax,miii,teinp.tenip2; 

rtirsoRi);  2in 

ysspeitig*x+30; 
xl=(  102-4— weigla.s*lower)/2; 
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plot9ode-di8play.count(<lupl«y.c) 


L..^  --'  i.V  ^  **  ^  ,» 


x2s(  lOS^— w«ghl.s*ti|)p«r)/2: 

Hndiiiax(MT«y,i:inM,i;niiii.iipp«*r  .lower): 
iaot>i<k>wer)  { 

if(iiiMk[i]3sl.){ 

colorjoff(0.0.0. 1  .iiodp(il ); 
bigj>loi<i*wei(siit.H‘Xl  .y.weig||t_»/4*3); 


loo(»j(iipper)  { 

iltmMii.il p[i]  as  t.0){ 

aet.color<  niikx.niiii.atT«y(i][i] ); 
drttwiM  X I  -t-weiglit.8*i.y-f  weiglit.a*3/4 . 
x2+wieiglu.8*j.y+«pcing):) 

) 

} 


) 

folorh*ir(l024-2.'i«.y+l0  .nwx.min): 
riirsonO;} 


fiiidnMx(amiy.niaxjiiiii.ouU.im) 

iMT«yQ(«M).*mitt.*miii; 
iut  oiiU.iim: 

{  iut  ij.k; 
bit  maxisO.inwgaO; 
iut  niiiiisO.ntiiu— 0; 

•mill  a  «iTay[0][0); 

•max  a  arr«y[0](0]: 
loopi(inii){ 

loopj(ouU){  if  (iirrnyC]p)<  •min)  •minsMr»y[j)p]; 

if  (•rr«yp]p|>  •max)  •maxsarrayplp]; 

}) 


diaplay.cnunl( ) 

{  bit  wlawr  a  rtSO; 
linewidthd); 
colorlT); 

rwlfK  m)0.  wlieie.923,  whfTe-f20); 
rolor(!i00); 

rerti(800.wliere.923,wbere^30): 

rok>r(  1); 

write.mt(905.wlierH-2, "".count);) 
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clMck.iiiousa(grapiuc.c) 


DATE,  f  OH  l9St 
VERSION: 

N.iilE:  Grmfkie  P»rk«fe 
MODVLE  NUMBER:  I.IO 

DESCRIPTION:  Ortpkir  Romitn*  for  ikt  Siltroo  Crophit  IRIS  -il-iO 
.ALGORITHM:  Sant 
PISSED  VARIABLES.  None 
RETURNS:  Sont 

GLOBAL  VARIABLES  USED:  Nont 
GLOBAL  VARIABLES  CHANGED:  Nont 
FILES  READ:  None 
FILES  WRITTEN:  None 
hardware  INPUT:  Nont 
HARDWARE  OUTPUT:  Sont 
MODULES  CALLED:  Nont 
CALLING  MODULES:  Nont 

.AUTHOR:  Grtgarp  L.  Tnrr 
HISTORY: 


#iiMhKU‘‘gl.]i‘‘ 


10 


!0 


#iuehide*'d«vlca .  h" 

#ia«htd«  an 

«<xtoru  Sunt  liMioryQ; 
axt«ni  iut  (ouui.<lwi>la>-.<iojMn; 
ntt  fmi  3s  0; 

fhnrk.mowM)  fllPtkjUOllSe 

{  nbort  val: 

float  tniiipj((nHi|>.)'tntiip; 

SrrPWMOOrd  *».>••:  *u 

iut  whkhx.wUKhy; 
dtar  nif«n|i«(20]: 

if  (qto«t()!aO)t 


xnsgrtvalnatort  klOI'SEX): 
y*sg<>t\-«liiatori  MOUSEY); 

x»  s  (itttK(flnMt)X8  /  niino): 

yn  s  ( iut )<( float  )y*  /  riilvo):  im 

which*  a  (M— 30)/100: 
whkhy  a  *n/S0: 


Nor  .10  ms 


Pngr  t  of  fmphif.r 


ch«ck.iiiou*«— blowup(graphic.c) 


•witdi  (,qrr«d(fcv8J)){ 

owe  MIDDLEMOUSE:  iflvalsslX 

bio«rnp<xs.y«):} 

vIm  ( 

r««iar«.*rr«Mi(): 

el«w.serwi():) 

hr«Mk: 

CM)  LEFTMOt'SE:  iflvsIaalX 
AiidiKHM 

braak:  ) 


CUM  RIGHTMOUSE:  iflvalaalX 


if  (y,  >  6a5){ 

•witc!a(whiclui){  • 

CM*  ‘4;(Mt  s  l-.lnr*^; 

CM*  3:«xiM0):br*Mk; 

CM*  2;in*nn();br*wk; 

CM*  l:in*k*jr*pb():hre«k; 

CM*  U:'wliil«(v»iaal)  i|reml(A:v*l); 
wluln(v«l!sl)  qrea«l(3cval): 
break; 


br*ak;| 

1 


DISPLAY.NETO, 

) 

) 


hknrtipixs.ys) 

Scioncoord  w.ys; 

{SfwnroorJ  xf.yf, 
riirsofll ); 


HU 


J3:O0  ffon  iO  1938 


P»ff  i  of  inphir.c 


bknrup-iiialce.^rapli(grapliic.c) 


xc»8HvtUiiiUor(MOIiSF;X); 
yc3getvalu«(or(  MOUSEY): 

viewport!  xc- 100  JIC+ 100.yc~  100.yf+ 100): 
liiiewi(llh(2);color!512); 
refti(x»-05.y»-95,x»+05.y8+95); 
ailor!0);rl«p»r! ): 

oftho2((Cooril)x*— 20..(Coord)xa+20..(Coord)y»— 20..(Cooftl)y*+20.): 


} 


mtore.«creeu( ) 

{ 

viewport(0.(iutKIU23.  *  vi«ieo|.  0  .(uit)(~<17.0  *  video)): 
oflho2(0..1023..0..76T.): 

) 

eolor.of)  hi.kt.  value) 

floMt  liijo.value; 

{  coMINDETCRM); 

m value  >  hi)  eoMHARO.ON); 

iRval«ie  <  lo)  color! HARD.OFF): 

} 

/*  «09.900.410.6*0  •/ 

nmliej(ra|«h!x.y) 
float  x.y; 

{  iut  i.ilidex: 
char  miiiih(!i]; 
ctiraoin ): 

if  !couiit  <  display)  retom; 

viewtior4!!hit)!x  *  video),!iiitM!x  +  .100.)  •  video). 

!iut)!y  *  video).!bit)!!yl-150.0)  *  video)); 

iflcbniit  <  10000) 

ortho2!0.,!float)couiit,  -0..1,hiatory(0]): 
cdae 

ortho2!!float)cotint  —  I0000..!float)count,  — 0..5.hittory(0]); 


JS:O0  Sor  SO  mO 


f}0 


restorc.scrccu 


lan 

color.of 


l«l 

luake^^apli 


\W 


Pf^9€  .1  of  pof^ic.c 


make 4(rapi>-colorbar(grapluc.c) 


colorf  1000): 

cImH):  •*" 

raloH  1001 ); 

index  a  raunt/(lM|ilsy; 

loopi( index ){  /*  korKomt»l  */ 
nNve2UfkMt)(i*dwplay),-0.5). 
drmw2((fliMit  M  i*dMi>lny  ),hirtory  [0] ):} 

iKilo.Mu  's  TRI'E) 

{  index  s  (iut)lii«tory(0]:  iro 

loopi(  (Unlex)  ){  /*  ftrltrlt  •/ 
niove2(-0.5.(Sant)i): 
drm«2((Raet  )co«ttt. (float  )i);]  } 

<olor(\'ELLOW); 

imiex  s  count/di*|d«y: 
iiK>«-e2(0,liHiory  (0| ): 
loo(>i(index>f  1) 

dr««2((fla«t)(i*dMpl«y).hwtoryri]):  im> 

r<atoiv.srfeen( ); 

irri(e.lkMkM(iBt  )x.(iat  )y-)- IM.M.—.birtoryfOi. 1000): 
wriieJlo*t((iat)x.(ittt)y»-90.S0,"".liiaofyrindex],l000): 
write.iitring3((i8t)x  •l-50,(iBl)y,130,*'trm  lintary”); 
ninonO; 


iw 


rotorlHw(x.y.nMXjnin)  CulorljAT 

tat  x.y; 

float  niax.min; 

(  iut  i; 

chnr  niwortring{20].iniiittrins(20];  »o 

X  -s  10: 

far  (ias*;i<TABLE;i+4’ .!++){ 
rolar(i): 

W|a*lot(x+i/2*(512/TABLE)-20.y.l*.(512/TABLE)):) 

•|inn(BnMUisirmg.”XS.M”.innx):  tin 


ii:0t  Hup  10  tm 


Pft  4  of 


i|>riiiki[iiiiMiriug."X3 .  St"  JMin); 
«>lo*<  YELLOW); 


eolorbar-bigj>lot(graphic.c ) 


rniov2i(x4'20,y); 

rhM»tr(nuinirin(); 

rinov3i(x-»-ltS0.>-); 

rbaniTtniusiring); 

} 

art.raloH  mM.mia,  value) 
float  iiiax,nun.vaiue: 

(  float  permit; 
iut  roix; 

iflvalue  >  max)  value  =  max; 
iflvalue  <  min)  value  s  min: 

permit  a  (value— niin)/(max— min)  *  (floatKTABLC— 0); 

coix  a  (iiit)permit  -f  8; 

rolor(colx); 

} 

#if  FALSE 

biU>l<A(x.]r.<)at) 

iat  x.y.<iot; 

{ 

(flout  )(r+M/  S).  (flout )( t+4ut)); 
irumt( I.  f.T.  f+uiztjufjiot); 

V 

1 

#«Kiir 

l>iU>lot(  xor.ww.)ib) 
iut  x.y.ww.hli: 

{ 


«n 


set.  color 


140 


big.plot 


390 


liig.pU>t 

MO 


J1:0C  Sou  JO  toss 


Puff  S  of  frufhir.f 


bigj>lot-textptl(grapliic.c) 


(llo«»)(x+ww/2),(fc**My+«»»n; 

} 

I  Jr»wit(x»Urt.>^rt.wtKl.]W«cl| 

int  xjitwt.y>iUrt.x«Hi.y«i«<-. 

'  „wv*2((«oia)xMMt.(ao**)y»twt»; 

<lraw2(  (00**  )x«mJ.(6o«»  )y«Kl ); 

,lr*witi(xM«rt.y»urt.x«nl,y*ni<«) 

i  iut  x«u»i.y»i»»t,xMnl.y«id; 


.  J 


2Tn 


tlrawit 


Hi 


no 


tlrawiti 


I 


niovrti((floi»t)x»urt.(0o*i)y«urt); 

<ira«2i<(0(Mit  )x«Hl.(0o**)y*®«*>'' 

dr»wii2(xrt»ri.y»J«tje-0J.y«Mll 

OoRt  x»ti»rt.y*Urt.x««d.jwn<l: 

hio»«2(  xxMrt  ,y»t*ft ): 

draw2(  x«»d  .ywJ ): 

) 

{ 

ciinofl( ): 
roMO): 

IrxlpilO: 

runonO:) 

trstptIO  /*  V»€^  fur  V 

{ 

l,xt|H>rt(20di«tM4W*videoMnrtM400^klro)dlnt)(-00.Sid« 

piigfroMIUUUI: 

"^'AN ); 

»y»t*-nd"el*Mr");} 


clra'ivit2 

3»t 


flcar.screou 

90t 


J 


textptl 


1 


}}:90  It  Of  30  1900 


Pot*  6  of  frofkir.c 


1 


t«xtptl-iiut.8creen(graphic.c) 

iuit.dav 


f 


¥ 


iait.il«v( ) 

{ 

gbrginO: 
tMiore jirrmi( )  . 
gcoiifigO: 
textptiO; 

) 

iiiit.ncrwiio  luit.screeu 

{  ittt  ij.k.tc<i.grwtt.biu«; 
gbrgiiiO; 
mtorr j(rrpni( )  : 

330 

gronUgO: 

nui|Kolor(  1000.100.100.100): 

lll•(><oior<  IOU1.50.riO.I)0); 

tfsiplK ): 

float  .*<rorn( ); 


r 

em0rSi(4SO.6.Ul); 

fk0r»tr('Act0*l'‘); 

fm«r3i(SS0,6S(l); 

fk0niir('G»tM~): 

cmofiif 6S0.  (SOJ; 

fk«nfr('litfkl‘J; 

•I 

q<lo>  M  MIPDLEMOl'SE); 

<l<lovKo(LEFTftlO('SC); 

(|do>ifo(  RiUHTMOUSE). 

for(i=TABLE.k=0j=«-J«.TABLEg++){ 
if  U<(TaI}LE/2)) 

{l(rom3(iiit)((250./TARLE.)  *  (fla«t)i)*2; 
liiuo  s  2jM-gt«en  ; 
nwipcotor<j<04r«on/3.Muo);} 

«iIm  { 

ml  3(iut)((3«(l./TABLE.)  *  (floMtMj-TABLE/3))*2; 

gr»«i3(250— hliH*): 

im|Kolor(j,ml,gToon/3.0);} 


340 


300 


.KM 


1 

<lo.»frooii( ); 


-J 
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P»f€  7  of  yropkir.r 


1 


•IT 


u»it.sc»«ei»-writ®.»tring2(grapUic.c) 


1 

(k>jirTcra( ) 

{ 

write  •triii«2( 30.700. 10,"IALT«). 
write  utring-Ji  1.10.TOO..M.-aUPl'‘); 
write.siring2(23O.T0O.-tO,''linnr ): 
wrile])triag2( 330.700.40.  “out*'); 
wrile.«itriii«2(430.700.40.-FMt-):} 


.do.s<Tecu 

3TI 


write.iitriiij2(x.y.l.titte) 
iut  x.y.l: 
dmr  title(23]; 

{  rluur  niiinber(20]; 

I  s  (iut)((6uNt|l/vidva): 

folor(.'i): 

rertfi(x-5.y-5jt+l.y+lS): 

ro(or<'1): 

linewitllbi  1 ); 

re<ti(x— "i-y-Sjc+l.y+lS); 

rnio»2i(x.y-2V. 

Hiiintr(title): 

} 


wTit«*.strmg2 

3«U 


3!itl 
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CHECK.ERRORS(tent.c) 

• 

•  DATE;  f  Ocioitr  1989 

•  VERSION:  S.O 

• 

•  NAME:tf$l.c 

•  MODULE  NUMBER: Lit 

*  DESCRtPTION'.Pnndt*  far  tfUm  Its!  am9  tvaimaliaa  dariaf  Iraiaiaf 

*  ALGORITHM:  Nomt 

*  PASSED  VARIABLES:  Noat 

*  RETURNS:  Namt 

*  GLOBAL  variables  USED:  Norn, 

*  GLOBAL  VARIABLES  CHANGED:  Namt 

*  FILES  READ:  Nam* 

*  FILES  WRITTEN:  Naat 

*  hardware  INPUT:  Nam* 

*  hardware  OUTPUT:  Nomt 

10 

*  MODULES  CALLED:  Namt 

' 

*  CALLING  MODULES:  Nomt 

* 

•  AUTHOR:  Grtfarf  L.  Tarr 

•  HISTORY: 

• 

20 

ext«ini  mt  ra<int.right,goo(i.t«at,muii.te*t.exMiiplsnjMiiiple,ctypr; 

extant  int  ilw|>lay; 

extent  ttntet  nciiral.net  net: 

iitt  (lertMomi: 

fkint  nrrurnry; 

#mein<te  "deftaltiewi.h'' 

#iudncle  "met.k" 

#iitclncle  <UMtli.h> 
iut  uiintakea  =  0; 
tkiet  liwlory['W)0]; 

float  errorfiiite]; 

extent  ntt  aulput.iiipnt.hide.oue.hidejero; 

30 

C'IIF;rK.ERRORS(y.don) 

float  yO: 

CHECK.ERRORS 

float  lionQ: 

{  btt  ij; 

iat  rorrert.rightsO; 
int  comet ^ootlsO; 

++net  .f  lnnti.connt(f  ty  pe) ; 

loopi(o«itp«it)  {  errurfi]  =■  ( float HMlCi]—  y[i]; 

if  (error[t]  <  0)  errorp)  =  — errorp): 

40 

if  (errorp]  <  0.5)  (correet^ood+-f: 

«leei*ion»++;) 

if  (error]tl  <  0.2)  rorreet.right-f-f: 

) 

if  (forniTtj|oo«l  ==  (o«ttp«t))]  gooU-f-t-: 

-f-fiiet.rlaM.te«t(ctype|:  ) 

VI 

>i:0l  Nav  30  1989 

Pa9t  1  af  Itat.r 

CHECK.ERRORS-calculate.error(teat.c) 


elM 

if  (inistakM)  priiilfl'‘S««pl*  XSd  Typ« 


XdKa**  3*mple,ctype); 


if  (corwrt  .right 


) 

float  «r,oldm*0.0.J<frvm=0.0; 


(output))  riglit-f-f: 


cnECK(y.doft) 
float  yQ; 
float  doflQ; 

I  float  iiiiMnl.hysO: 
iut  ij; 

loopi(output)  { 


) 

niiwu^J.hy  =  0.; 


niMMKi.by  =  (float  )<loft(i)—  y(t): 
m  +=  miawd.by  *  tni<«rti.by; 
/•  fnnifrdoft  %3.If  t  %i.lf  ' 

MfilihN):  •/ 


) 


rl^ar.t^( ) 

{fait  i: 

loopi(output )  {net.cla*«.t«t(i]sO; 

uet.rlaM.countfi]  s  0;) 
right  =  0:good  s  O.tWiaiou*  s  0; 

) 

DO.TFSTO 
{  fait  i: 
clrar.t«st( ); 
loo(>i(<']i»m.l»at){ 

M AKK.TEST(nrt  inp.iwt.doft.i): 
fBED.FORWARtX); 

<  HEt  'K.ER  RORS(  iwt.outp,ii<ft.iloa); 

printR-Taat:  «.W  X3.M  \a".(float)right/(float)ex8in.t«t*100.. 

(float  )good/(float)exani.t«»t*  100); 

cl«ir.t«it( ); 


) 


ralciilalr.rrror) ) 

I  fait  ij.inHrx; 
••rrs  0.; 


rl^»r.t«it( ); 
ioopi('*xamplftra){ 

MAKEJNPt'T(n<M..inp.ii«‘t.doft,i); 

FEED.FORWARDO; 

( il  Et’K(uM.oHtp.nct.doft); 


) 

m  s  (float)aqrt((<louble)erT): 


IHI 

CHECK 

I 


TO 


clear.tpsf 


DO.TEST 

«( 


90 


c  alciilate.error 


t<MI 
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r*ff  3  of  U»t.r 


calculata.error— CHECK2(te8t.c) 


ilerverr  =  (m  —  otd^rr); 
oliWrt  =  err; 

iflexMiipInrs  ==  0)  err  =  100.  —  accnrxry; 
iiMlex=rouiil/di»pl»y; 
history  [index]  =  err; 
err  =  0: 

) 


<IIECK2(y.iloa) 
fluMt  yQ: 
fluAt  <lo(iQ; 

{  SimU  misied.hysO; 
int  ij; 

loopi(output)  ( 


) 

iiiined.hy  =  0.; 


inissed.by  =  (float)<lo(l[i]—  y|i]; 
err  +=  niMsed.by  *  luissed.by; 

} 


110 


CHECK2 


i30 


Ji:0J  i\or  10  toss 


P»ft  1  of  Irsl.c- 


nudn(couater.c) 


DATE:  n  A»g  1988 

VEESION:  1.0  • 

S.iME:  Counter  Propugution  fifeural  Set—Moin  Loop 

MODVLE  SVMBER:  1.0 

DESCRIPTION:  Counterprop  (n«m  loop 

ALGORITHM:  None 

PASSED  VARIABLES:  None 

RETURNS:  None 

GLOBAL  VARI.ABLES  USED:  None 
GLOBAL  VARIABLES  CHANGED:  None 
FILES  READ:  None 
FILES  WRITTEN:  None 
MODULES  CALLED  None 
CALLING  MODULES:  None 

.AUTHOR:  Gregorg  L.  Tnrr 
HISTORY: 

. . . 


#iuclu<le  "aat.h** 

#iiiclu<le  "dafiaitloM.il** 

#(l«sfiue- wail(A)  if(((ouut9(.<A))s=0) 

•tmct  iwuroJ.nat  nrt; 

int  iiiri«it.output,hi(l«.oii^30; 

extern  suniple; 

int  display  = 

extern  int  nienu(); 

int  rouiiisO.right.good.guess.test; 

iiiaiii( ) 


INITIALIZE! ). 
normalisejlatal ): 
while  (count  <  lUOOOO)  { 

/•  irttk  counter  propngotion  •/ 
MAKEJNPI'T(net.inp.net.yont,-l); 


•1:0!  Nor  -10  1988 


Page  1  of  counter. c 


tiiaiu(  counter  .c) 


FEED.FOmVARD(TkAIN ); 

TRAIN.NETO: 

/*  nmopt  roaairr  froftfation  to  ttti  */ 

MAKEJNPnT(nrt.iitp.n«'t.yoiit,— 1); 

FEEl).FOR\VARD(tHECKII);  «ii 

C'UE('K.ERROR.S(ii*t.yp.iirt.yout); 

wMMiJMplay)(  SHO\V(); 

DO.TESTO; 
line;  } 


waiMdisplay)  DISPLAY.NtTrO: 

display_couat( );  td 


)l 


12:01  S«r  to  1988 


Pafe  2  of  fooolfr.r 


(defliiitiona.h) 


• 

*  P.\TE:  U  .Uf  I9SS 

*  VERSION:!.  I 

* 

•  SAME:  Drfi»tHon».k 

•  MODULE  NUMBER: 

•  DESCRIPTION: 

•  ALdORITHM:  Not  i« 

•  PASSED  VARIABLES:  None 

•  RETURNS:  None 

•  GLOBAL  VARIABLES  USED:  None 

•  GLOBAL  variables  CHANGED:  None 

•  FILES  READ:  None 

•  FILES  W  RITTEN:  None 

•  HARDW  ARE  INPUT:  None 

•  HARDWARE  OUTPUT:  None 

•  MODULES  CALLED:  None 

•  CALLING  MODULES:  None  Jo 

• 

•  AUTHOR:  Greforf  L.  Tort 

•  HISTORY: 

• 

#<k«ua  GRAPHICS  TRUE 
video  O.tfO 

ijldHIiMi  loopi(ON)  ftwr<i=0:i<ON:i++) 
loopj(ON)  for(j=0u<ON-j-H') 

#defliM  loopli(ON)  for(ksO;k<ON:k-fd>)  m 

#defliie  loo|»ij(ONE,TWO)  fiir<i=0;i<ONE:i++)for<j=0J<TWO-j4-+) 

TABLE  513 

#d«>ltiK<  HAIID.ON  RED 
#<leaii«  INDETERM  GREEN 

uard.off  blue 

#<lofliio  TRUE  1 

#<l«dtue  FALSE  0  m 

#doflne  TRAIN  0 
#deHiM  CIIECLIT  I 


#d(iflu«  line 
#doBiM  diaply  100 


2i:0i  Nov  iO  1939 


Poft  I  of  iffin$Uon».k 


aUe  300 

struct  nrursl.iiet  { 


(n«t.h) 


W  yo«it(«ic];  /•  .VOOf  V  omtptH  /•••■I  •/ 
n«mt  y|i(«ttp];  /•S’ODE  >"•/ 
flo«t  yi[8ise][Msc];  /•  WEIGHTS*/ 
fhMt  typ(sisc](sisc]; 

flout  /•  A'o/Jf  2*/ 

fluot  I.ronjsiip]; 

floot  xpfsucj;  /•  SODE  A"  V 
flout  u(sise](sisc]: 
float  zxp(sisc](sisc]: 
floot  inp(tisc];  /•NODE  A'*/ 


3tl 
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INrnAUZE(iiiitialize.c) 


DATE:  J  Aafmti  I  US 
VERSlOlt: 

NAME:  ImtiitlisMion  Modutt 
MODULE  NUMBER:  tS 

DESCRIPTION:  lnUiMi:Mian  of  tkt  diofitf  kordmon. 

«af  iofoi  left  dot*,  omd  net  dot*  otnctnrr. 

ALGORITHM:  Noof 
PASSED  VARIABLES:  Nomt 
RETURNS:  Non* 

GLOBAL  variables  USED:  Ntt 
GLOBAL  variables  CHANGED:  No*t 
FILES  READ:  Noot 
FILES  WRITTEN:  Noot 
HARDWARE  INPUT:  Nome 
HARDWARE  OUTPUT:  Noot 
MODULES  CALLED:  Noot 
CALLING  MODULES:  None 

AUTHOR:  Grtfon  L-  Tott 
HISTORY: 


#iucln<l«  "dailaitloia.k" 

#iarbid«''fl .  k" 

#iiicladc*'4«Tic« .  k" 

#iuclnfle  <stdio.h> 

#ia«lu<l«  <w(n«l.h> 

#iiich«<l«  <nvUh.h> 

#iu«lnd«  "■•t.k” 

«XtMru  atmct  itrt; 

pxteru  iu*  iii|>ut.o«t|>ut,hi<h*.oii«; 

flodt  nuHionK): 

#<leflii«  TABLE  512 

iut  inninO; 

INITIALIZEO 

{ 

/*  sifool  olio*'*  lk(  mto*  to  kt  roUtd  kf  o  roofrol  r  */ 


S1:0S  Nor  SO  1988 


■to 


V 

90 


4n 


INITIALIZE 

%{ 


P^ft  i  of  mthmlizr.c 


INITIALIZE-init.net(itiitiaUxe.c) 


► 


» 


I 


» 


t 


r 


•ignal(SIGINT,inrau ); 

iiiitjiriwii( ); 
init.d«ta( ); 
initjiet( ); 


initjM'r««nx( ) 

{  iut  ij.k: 

'vinop^n("**t"): 
ginitO: 
groniigC ): 
roJor(  BLACK); 

liiipwidth(  1 ); 

fok)f(  YELLOW): 
riiiov2iHO.dOO); 

chMStrCCovatar  Propagation"); 

(liMV  21(40.550); 
chantr("Ioaral  lotaork"); 

nia|Kolor(GREEN.50.50.50); 

for(  ixTA  BLE/2.kx0  jx«a<TA  BLE;i4-f ) 

{  if  (j<2M)  map<olor(j.0,l2S— i/2.i);} 

if  (j>2tf4)  ik++;  iJ»8prok>r(j.k,J28-k/2.0):) 

) 

} 

inii.nrt( ) 

{  ■ 


iut  ij: 

loupijiotlipiit.hidr.one) 
wi.*>p[i)[j)  =  1.0; 

nonnalii«.w«igii(a<n«i.iyp.ou(pu(,iiidp.one): 

loopii(ottlpu(,hidr.on^) 
nM.yi(i][i]s  raiidoniO: 
noriiialii«i,w<Hghls(  nrt  .yz, output  .liide.one) ; 

loopij(  hidr.oiiv.input ) 
iHH.xz(i][j]  s  1.0; 

noniialia«.wrighla(n«t.xa.bide.oii«,inpiit): 

loopij(hidp.on^.inpiit ) 
ii«H.zxp(i][i]  =  1.0; 
loopi(hidr.onv)  inrt.z.roii(i]  =  0.0; 
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m 

iuit.scrf*eux 
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iuitjM't. 


too 


PBft  f  of 


iiiit.iiet-bigj>lot2(iiiitialise.c) 


printrcni^iglita  loxMaixadXn"): 


iK>niwliie.««ig|iis(wvighta.iipp«r.k>««r)  - 

float  weigiitiiO: 
iut  U|>p«rr.knm; 

{  iut  ijsO; 
float  iiukg: 
loopj(upp,?r){ 
iiMK  =  0.0; 

loopi(k>a«Y)  nmg  +=  <rngli(434j*aisi>)  •  unghto{i-fj*MXp]; 
mag  =  aqrMmag); 

loopi(lo«n)  ir«tghta{i+j*usr]=  w«sh»^i+i*»»el/mag; 

) 


float  random  () 

(  float  x; 

iut  y: 

ys(rM,l()  %  1000)  . 
xs((float)y/l000.0-0.5): 
return  x; 

} 

flndiKxH ) 

I) 

I>igj>iat2( ) 

{} 


liu 


uoniuJize.wcights 
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fiutluode 
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FEED.FORWARD(feedforwtird.c) 


DATE: 

VERSION: 

NAME:  Fttdforwari 
MODVLE  NUMBER: 

DESCRIPTION: 

ALGORITHM:  Nout 
PASSED  VARIABLES:  Non* 

RETURNS:  Nout 

GLOBAL  VARIABLES  USED:  Nout 
GLOBAL  VARIABLES  CHANCED:  Nout 
FILES  READ:  Nout 
FILES  WRITTEN:  Nout 
HARDWARE  INPUT:  Nout 
HARDWARE  OUTPUT:  Nout 
MODULES  CALLED:  Nout 
CALLING  MODULES:  Nout 

AUTHOR:  Grr§orf  L.  Torr 
HISTORY: 


’‘dcfiAltloma.h" 

#iiichida 

#fai€hi«le  <iiwlh.h> 

atrnrt  n^-. 

iut  in|iiit.ouipul.liid<>.onr.co<iii(: 

int  tn«xjn<i^x; 

FEED.FORWARD(Ut> 
iut  tai; 

{ 

iut  ij; 

flout  imx,t«inpl.t«tiip2,bi; 

/•  rokuIoU  :  t  */ 

/*  Notf  tkoi  tkt  fttd  forword  ulforUkm 

»ork$  a  link  difftrtnilf  rktu  oeitollf  Ustiuf 
tkr  ofottm  (M  ofitoofd  to  tturkiuf. 

For  tfurkiuf  ikt  foul  rolut  ort  cotultr  fro/iofoUd 
Ikrou^  tkt  tfottm.  For  Uttiuf  foot  rolut  on  ottomtd 
to  kt  ztro. 

Tkt  ttM  tonditiou  otoUmtut  m  tkt  dookk  loop  tktu 
orcoouts  for  tkis  dtfftnott. 


V 

loopi(liHla.ona)  (l«nipl  =  0.0;Ufnip2=0: 


ii.OH  Nor  SO  1 900 


!0 

V 


3n 

FEEDJORmnD 


4(1 


SO 
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FEEDJFORWARD(feedforward.c) 


loopjCinput)  traipl  +=  net.inpQ]  *  iiel.xi[i][j]; 
iqtiii  Is  TRl'E  ){ 

laopj(outpul)  t«iip2  -fs  nH.youtli]  * 
tHrt.i{i]s  tempi  -f  t«nip2: 

) 

/  *  Now  find  mnx  of  j  i  */ 
max  3  O-.msxJmlex  s  0; 

loop i(  hide  ooe)  {  W  s  uet.i.eon(i]/(fl(Mit)ro<int  -  1.0/(fliMit)hide.oue; 
if  (Ui  ss  TRUE)  U  3  0.0; 
if  (hi  <  0.02)  { 
if  (net.xfi]  >  max)( 
maxjndexsi; 
max  3  net.xfi]:)  )} 

iRtat  Is  TRl'E)  iiet.s.roa(niaxJudex}  -t>s  1.0; 


/*  if  ooiuf  ou  0  >  ipniol  drfoaU  ia  to  fkt  lowtti  tndrr 

<M  nrammrndti  if  Bttki,Si*Utn  */ 


loopi(hide.oiie)  {  if  (issinax Judex) 


iiei.s(i]3l.0; 


iict.s(i]s0.0;} 

loo|ii<<Hitpiit)  (net.yp{i]30.0; 
loopj(lii<le.oiie) 

{  net.yppj  +3  net.»yp{j]nj  *  net.s[ii;}} 

loopi(inpiit)  {net.xp[i]s  0; 
loopjlhide.oue) 

{  ueijip(ij  +3  uet.xxp[i][j]  *  net.xd];}} 

) 


li.Oi  Sor  10  1909 


Poft  i  of  fttiforwotd.c 


J.L: 


TIlAIN.NET(trainnet.c) 


VERSION: 

NAME:  hnim  ari.r 
MODULE  NUMBER:  i.4 
DESCRIPTION: 

ALGORITHM: 

PASSED  VARIABLES: 

RETURNS:  Nout 

GLOBAL  VARIABLES  USED:  VtfUr* 

GLOBAL  VARIABLES  CHANGED:  tVtisht  Vecion 
FILES  READ:  Nont 
FILES  W  RITTEN:  Non* 

HARDWARE  INPUT:  None 
HARDWARE  OUTPUT:  None 
MODULES  CALLED:  None 
CALLING  MODULES:  Mnin  Loop 


AUTHOR:  Greporp  L.  Torr 
HISTORY: 


#defliM  THETA  FALSE 

#mclnde 

#iucliule 

ext«ru  ttrnct  ii«urai.iN>i  n<H; 
ffxtnm  iut  iiiput.OHlpul.liidv.one; 
#dtilliie  alpha  0.0 1 
0.01 

extvrn  iat  inax.ind«x; 


TRAIN.NETO 

{iut 

j*  ndjnoi  the  n<efpkt»  heimeen  Ike  p  prime  ond  Ike  z  toper  */ 
loopi(  output) 
iMfi.iyplinaxJmi'xKi]  += 

(—alpha  *  nrt.iypImaxJnJexIfi]  -f  b«ta  *  nrt.yoat[i]); 

/*  odjnni  Ike  treipkls  kelmeen  Ike  r  prime  ond  Ike  z  toper  */ 
loo|>i(inp*it) 

ii«i.xxp(iiiaxjinkx|(i)  += 

(-alpha  •  n«t.(xp{inaxJu<lM][i]  +  brta  *  nH.inpfi]): 

/•  odjnot  rnpkto  kelmeen  z  ond  r  tnpnl  •/ 
loopi(  input) 

iHft.xzluiaxjiKlfxJti]  -fs  alpha  *  (iirt.inp(i]  —  nft.xxtnwxjndrxKi])  : 


Ji:04  Nor  10  im 


v> 

V 


TRAIN.NET 

4n 
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TRAIN.NET(traion«t.c) 


/•  •Jjmtt  wtifki*  ittwttm  2  •nW  f  fftme  */ 

loopitliHk.one)  ,  ■  , 

iiet.>-*(iiuuUii<fcx)(i]  +=  brt«  •  (iK-t.youtp)  -  iHrt.>-i(ni*xjwMl»)): 


} 
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■howinput(8how.c) 


* 

•  DATE:  1  Oflobtr  t9SS 

•  VERSION:  S.O 

• 

•  NAME:  Show.c 

•  MODULE  NUMBER:  S.5 

•  DESCRIPTION: 

•  ALCORITBM:  Nom,  lu 

•  PASSED  VARIABLES:  No»t 

•  RET  'R  NS:  Nonf 

•  GLOBAL  VARIABLES  USED:  Nome 

•  GLOBAL  VARIABLES  CHANGED:  Nomt 

•  FILES  READ:  Non^ 

•  FILES  WRITTEN:  Nont 

•  hardware  INPUT:  Nome 

•  hardware  OUTPUT:  Nome 

•  MODULES  CALLED:  Nome 

•  CALLING  MODULES:  Nome  Iti 

• 

•  AUTHOR:  Gniort  L.  Tarr 

•  HISTORY: 

• 

#hielnda  <st<l»>.u> 

#turliule 
extern  int  diiit>l»y-. 

extern  atmet  iirutal.net  n<>t;  .to 

uxUwn  int  ia|>ut,oul|>al.tii<lr.our.ri$ht,go<xl; 

flout  DO.CUECKO; 

extern  iut  rount,rigbt,jums.goo<l.t«et; 

SHOW()  ■  SHOW 

{ 

«ho«output(nrt.youl,nrt.yp); 

r 

ehow»etyliU(mei.fz.l.omlfat,kHe,one):  *o 

skowno4((me1.z,ki<l(_ome  ); 

skow»rifki$(net.T:.l.kUf.0He.impat): 

skovimpatl  met.  imp): 

ohoirnodelmet.fi.kUfJwo): 

node  imfo( SOO.  600): 

V} 

iilio«!nput(s)  shotviupitt 

fleet  xQ; 

{  int  j.i:  $n 

priiiircCeuator  Prepayatloa  Nodal Nb"): 
loofiii input)  printf("  Z(Xd) 
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showinput-8howw«ight8(show.c) 


loopi(input)  priatf|"Xl-31  ''^(i]):iin«;liii«: 

■} 

»liowoiitpiit(yo<it,yp) 
flfwt  youtQ; 

Bwit  y|>fl: 

{  iut  ij; 
fluat  error; 

r 

loopi(omifml)  pnuljf('%J.Sf  ' ,po*iii]):hnt; 

prt»tff~Y-phmt  ~): 

loopi(omipmi)  priutf(“%S.Sf  '.tp(>]): 

V 

prin(f("Ce«at:X6d  \a6aM*:Xl.31  Good:  Xi-3f  Toot:  XI.MXa", count, 

(float  )ri§hl/(iMplay*  100.0, 

(float  )goo<l/(lMplay*100.0, 

U.O); 

hea«ler( ); 

/*  fpTiuif(*iderr,'Comnl:9(6d  Cut»»:9f-1.Sf  Good:  %S.if  Ttti: 

(fifijnpki/  diitpUp*tQ0. 0. 

(flott)toodl  ditpUp‘100.0. 

0.0): 


.sLowoutpiit 


•lio«node(  y,n.tiile) 

float  yQ; 
iut  ii; 

rJiar  tille(20]; 

{■  iut  i4;liiie; 

printf  ("Xs  ’’,litl<<);liiie; 

loopi(ii)  priiit(1("X2.4f  ",y(i]);line; 


sliowuocle 


allow  wei  gbU(  w,layer,iipper,iower) 

float  wQIaize]; 
iut  layer.iipper,lower; 

;  iut  j,i; 
line: 

loot>i( upper)  printfCVKXd.Xd)  ", layer ,i);line: 
loo|>i( lower)  { 

luopj( upper)  {  if  (w[i][i]  <  0.0) 

|irintr("Xl.3<  "■'^OK']); 


sUowwpigUts 
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• 

•  DATE:  J  Oriohtr  J9S8 

•  VERSION:  i.O 

• 

•  NAME:mfn».e 

•  MODULE  NVMBER:i.6 

•  DESCRIPTION:  Ptvridu  rnttrarUve  mrun  opUona. 

•  ALOORITHM:  Nout 

•  PASSED  VARIABLES:  None 

•  RETURNS:  None 

•  GLOBAL  VARIABLES  USED:  None 

•  GLOBAL  VARIABLES  CHANGED:  None 

•  FILES  READ:  None 

•  FILES  WRITTEN:  None 

•  HARDWARE  INPUT:  None 

•  HARDWARE  OUTPUT:  None 

•  MODULES  CALLED:  None 

•  CALLING  MODULES:  None 

• 

*  A  ('THOR:  Grtforf  L.  Tarr 

*  HISTORY: 

* 

extent  iut  ui.claa*; 
extent  iut  activ; 
extent  floet  Ihmliolil; 

#uichule  <siKnal.li> 

#iur.hi«le  <Kt(lio.li> 

#iufJiido  "deflBitiens.h" 

#iurlude  "aat.h" 

#iurhide  <kI.Ii> 

extent  atnict  neiiral.iiet  net; 

extent  iut  rouiit.riglit.good.guess.teit.display; 

iut  niennO 

{  cktir  *ele<-t.fileiianie(20): 
priiitf  ("  Hasa  \a"); 
printf  (”  1)  Sava  Vaighta'');line; 
printf  ("  2)  Diaplmy  Vaiglits'');line; 
printf  ("  3}  Kaad  Ualyhta  ''):line; 
printf  ("  4)  Chaaga  Act  Display" );liiie; 
printf  ("  6)  Shoe  Actlaatioa  Lavals");line: 
printf  (”  6}  Shew  Walght  LaTals"):line; 
printf  ("  7)  Chaaga  Display  Thrasheld'');liiie; 
printf  ("  a)  BXIT  \a"); 
printf  ("  SBLECTIOS:  ?  "): 

seit^t  =  getrharO; 
awitrJtfaelert)  { 
case  ’a' : 

gexit( ); 


(inenu.c) 


III 


50 


30 


40 


40 
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read.weightfl(ineiiu.c) 


.■xiMJ); 

br«ak: 

case  ’t':  write.weights  (): 

;break: 

case  '3':  prntf("\a«eights  Displajsd 

p*uss<); 

itreak: 

case  *3*:  priulfl "'VnEater  Fllsi«e: 

»canf( 

resd.«ri((lits 

priii»f("\»  t*  Weight*  lastalledXo’Vfilfimmi*); 
break: 

case  '4';  priiitf  ("XaBater  Cl***:"): 
break; 

ease  'S':  srtiv  =  TBI'E; 

nodniifo( 800,(400): 

break: 


case  'O':  activ  =  FALSE: 

ii«cW«ifo(800.<J00); 

break: 

case  ’7':  printf  ("\aBnt*r  lee  Threshold:"); 

Svauf<"X<"  tlliresliold): 

break: 


sigti(U{SIC*  INT  .menu ) : } 


read.weight*  (filename) 
dhar  (MenanieO; 

{  FILE  ‘fp; 

fl<Mt  x; 


#if  FALSE 
iut  ij.k: 

iut  foMlput.finput.flii'le.one.lhideJwo; 

fp  =  fopen  (filenanie,"r"); 
if  (fp=sNl'LL) 

lprintr("\a  •••«!*  Error***  \«"):exiM  I );} 

facanf  lfp."%d  W  U  Xd".4:foutput.&fl«ide.two,A:(hi<le.one,tftnpot): 


if((routput>oulput)| 

(fhide.one>lii<le.one)| 

(niide.two>lMdeJwo)| 

(fiiiput>input)) 


{priii(f("\a  ***ril*  Too  Largo***  \n");exiM  1);) 
loopiji  fmitput  .fliide.t  wo)  ( 
lscauf(rp."XT".4:x); 
net.w2(illi)=x:} 
loopiji  (liidej  tvo.fliide.one)  { 
f»c*nf(rp."Xf".4;x): 
iie».w)(i]ljl=x:) 
loopi  j(  ll  ide.one .  fi  II  p«i  t )  { 
ra<'auf(fp."Xl",&x); 

Det.w0(i)lj)=*:} 


tin 


T® 


M 

rcatl.weigUts 
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readl.w«!gbts-beader(  inenu.c) 


loot>i( (liidtfjwo )  ( iicBiifl  r|>,"Xl"  .u«>l .11 + i ) : } 

toopi(fhid«.oiie)  (CKuif(fp,’*Xf'‘.not.tO-fi):}  iio 

fclosrlfp): 

#<mdif 

) 

WTite.wvight*  0  writc.WeiglltS 

{  FILE  Tp; 
iut  ij.k; 

cluur  filMiiunr(20]: 

#if  rAL.se 

priiitf^‘'\BBatar  lilMuaa:  "): 

fp  =  fop«i  (filonMne."***); 
rprinlf  (fp.-Xd  Xd  Xd  Xd  \b" 
loopii(oiitpul,hi«(r.tt?ol 

fpriiilfifp."Xf  \i»".n«.w2(i)l}).ij): 
loopijl  hitle.two.liidvjMie) 

rpriiitnfp,'*Xf  \*".iHft.wl(i]C]jJ); 

loopijl  hidf.oa^.iiipiit ) 

fprintflfp.'^  \a**,iiH:.»rOp]p)jj):  lan 

loopi(oiit|>ul)  {fprinlflfp,*^  \a‘*.nrt.t2(i)):) 

loopi(hi<l«.two)  {rpriiilii[fp,'*Xf  \b". net. tip)):) 

lo<nM(hide.ooe)  (rprinttllfp,'^  \B".net.tOpj);j 

fcloaelfpltprintfC^  Veights  SteradVa**): 

#euclif 

) 


I40 


iieitdero  Ueatlcr 

I  iut  offx="80,offy=(K)0: 
color(  1000); 
liiiewidthl  YELLOW); 

nrtn(o(lx.ofl'y,180+offx,(iut)(70./vHleo)4offy); 

color)  I );  ito 

rerti(off'x.offy.l80+offx.(iut)(70./vi«leo)+offy); 

color)  YELLOW); 

writeJnl)o(rx+  I0.o)ry+ 10,  "Covat :  ".coiint); 
write,flo*t)olTx+  10.ony4-')0, "Right :  ",)fl(Mit  )gocMl/(lnplay*  100); 
wri(e^fioat)oflx+ 10, ofry470,"Qv«M :",)  float  Irigbt/dbplay*  100); 
write,float)offx+  10.offy+30.“TMt  :  ".test ); 

} 
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no«lciiiro<offx.off'y ) 
bit  oflx.oiTy; 

cok>r<it): 

ivrtfi(04<>ffx.0-foffy.l504«II*.80-(-offy); 
rok)r(  1); 

rwti(  O+offx.O+offy,  1  SO+offx  .80+offy ): 
rok>r(-l): 

rniov2i(  lO-foflx.S-^-fofTy): 
rliar>4H"Carr«:it  Olaplay**): 
ciiiov2i(  l(M-offx.4Q-H>fT> ): 
rh«nilr(''SheM''): 
rniov2i(  104ofrx.2S-H>ffy): 
if  (xctiv) 

rhMMr(  "ietlvMlaa" ): 
fllne 

rhanttr( “llalglit  Lavals**): 

1 

writr,flo»t(  x.y. title  .ft) 
bit  x.y: 
diar  titl«.‘(20]; 
fkMt  ft; 

{  cliar  auiiiher(20]; 

!<t>rinift  iiiiniher.'^ .  2f  ".ft ); 
ciiK>v2i(x.y): 
rliantrl  title); 
rli»rstr<iiuiiibef):  } 

writejiit(x.y.title.ft) 
bit  x.y; 
dmr  iitle[20]; 
bit  ft; 

{  duw  nuniber[2fl]; 
spriiitft  n«nil»t  ."Xd"  .ft ); 
riiiov2i(x..v); 
rharatrl  title); 
rh«r»tr|  number  I;  } 


header  -writejiit(nienu.c) 

uodeiufo 

140 
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write.float 

ion 


tt<n 

write  jut 
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DISPLAY.NET(duplay.c) 


• 

*  DATE: to  Atfust  1930 

*  VERSIOS: 

* 

*  SAME:  DISPLAY.SET 

*  MODULE  NUMBER:  1.7 

*  DESCRIPTION:  Pnrtdt$  dUpl^f  nuliufs  of  rouulerprop 

*  ALGORITHM:  Soot  lo 

*  P.iSSED  VARIABLES:  Sent 

*  RETURNS:  Nont 

*  GLOBAL  VARIABLES  USED:  Noot 

*  GLOBAL  VARIABLES  CHANGED:  Nont 

*  FILES  READ:  Nont 

*  FILES  IVRITTEN:  Nont 

*  HARDWARE  INPUT:  Nont 

*  HARDWARE  OUTPUT:  Nont 

*  MODULES  CALLED:  Nont 

*  CALLING  MODULES:  Nont  jo 

* 

*  .AUTHOR:  Grtforj  L.  Tort 

*  HISTORY: 

* 

#iucln«le 

#iucln<le  "dafialtlaaa.k" 

«xtc«ni  uit  (ounl.riglil.good.tMt: 

toettwu  nMirnljiH  iKt;  » 

octoru  iut  iiiixit.oulput.liidi^.one; 

iut  nctiv  =  FALSE; 


niSPLAY.NETO 

{ 

H  (!  Mtiv)  { 

plotnodcno.n^  2,bidr.oii«,"Z-I0DKS*'); 
plotiioUrri  I  ,ll••l.illp, input, "Z-lodM :  IfPVT"); 
pbtno(l<<l( l,n^..xp.input,''I'-fria«  lodM”); 
ploln<KW(2,nrt.i.hi<lr.on»."Z-l008S"); 
plotnodH(2.nrt.z.liide.oiw." 
ploliK>dt:r(.‘),iirt.yp.oulput.''T-Prla«  lodM"); 
pk>tn<Ml<*l(  A  .lift  .i.lii<|i>.oii«.  "Z-lodM" ); 
plotiiod«’li(  l-ii'l  .iiip. input): 


DISPLA^'.NET 
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i\or  W  19SS 


P«fr  I  of  iitMpU$.c 


DISPLAY.NET-plotnodeHdi8pl«y.c) 


l>loliio«M(  3.nft  .yout, output, ); 

» 

plotnodr  (O.l.ii«t.xxp,uiput.hi<l«.oa«): 
plotnode  (l,2jietJU.liiiir.oae,iupul): 

no 

plotiiode  (2.3.ii«t.yz.oiitput.lti(l«.oo«): 

» 

plolDode  (3.4.iiet.zyp.b‘de_oiie,output): 


I 


» 


» 


» 


I 


} 

J 

#<l«Aae  wfigiit.s  20 
j*  ISS  •/ 

#<kifla*e  «pcing  l-'iS 

floMt  lliKsiiolds  0.0; 

plotno<H<  s.iiodr  .hMwr  I 
iut  x.Iouky: 

0oMt  iknIpO; 

I  int  ij.k.y.x2.y2: 

y3sap<'ing*X+20; 
xa(  1024-wwghtjs*lower)/2; 
loopi(lo«n>r)  { 

»-t.c«>lor(  J.O.O.O.JKxl^fi)): 

bigj»lot(  i*  w«ghi>+x,y.  weight. »/4*  3 ); 

cok>r.ofi0.5,0.5.node{i]); 
b-  4)lot(i*weiglitji+x+  100.y.w«ght.»/4*3): 

coior.oRO.O.O.  l.iKxhrp]): 
higj>lot(i*wcight.»+x+200,y.weiglit.»/4*3); 


)) 

ploliKMmx.lKxle.lower.title) 

iut  x; 

flfMt  iiodvQ: 
iut  lower: 
rlmr  tilleQ; 


TO 


plotuodoi 

•  I 


plotuodrf 

ini 


I 
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plotnodef-plotno<leh(dwplay.c) 


I 


{  int  iJ,lt.yj(2o'2: 

.  ya*pdng*x-t-20-.: 

*=(  1024-i»»igfit.s*loww)/2: 

loopi<io««r)  { 

*rt_cok>r(  1 .05.0.0  Jio«l»(i]); 
bigj>lot(i*wrtgJii.»+*,y. 


) 

cohtitl}; 

rnwv2i(X'fl.50.y); 

diMiHlitlf); 

} 

l»lotiio<W(x.iiod*.low«.titl^‘ 

bit  x: 

fliMit  ikhI^: 
bit  lo«m; 
cbiir  titl«0: 

{  bit  ij.fc.ya2o'2: 

ys«p«-ing*(x): 

xa(  1024-w«ght.»*tew«r)/2; 

loopi(to«r«r)  ( 

•rt.col^  l.a5.0.0jKHl^i]); 
liigj;tet(i*w«ghtjM‘X.y,w«glit.»/4*3); 


) 


roiodO); 

<’iiiov2i(x+(50.y); 
chiintd  title); 


} 

plotiiodeh(  x.node.lower) 
bit  x; 

float  iKMb‘Q; 
bit  kmer; 

{  bit  iJ,k.yj(2.y2; 
yss(ieing*(x)-f  15; 
x=(  1024-weiglit.i*loerer)/2; 

loo|>i(lower)  { 

»et.rolor(  l.O.O.O.nodep)); 
l>ig4>(ot2(i*weight.s-fx.y.4.5); 


)} 
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J 


A 


plotnodeli-flndinax(dupIay.c ) 

pJot«xW(x.iio<fcJow«)  plotuodeil 

lilt 

iut  x,iK>(l<*(]  .lower; 

{  int  iJ.kjc2or; 

ys»|triiig*(x)-f20; 

x=(  1021— weightji*lower)/2: 

loo(ii< lower)  { 

rotor.onO.O.0. 1 .( float  )iio<le{t] ): 

l>igj>lot(i*weiglitji+x.  '*® 

y+weighl.8*2,weiglitji/4*3); 


)) 

ploti»o<M*.otlier.«rr«y.upper.lower)  plotUOtle 

iut  x.oiher; 
float  tkrrayQ[mte]; 
iut  iipper.lower: 

i«o 

{  iut  iJ.k.yjiljc2or2.yiip; 
float  ntax.min  ,tetiip.tefnp2; 
y=iH><'ing*x-f20; 
xlss(  1024-weight.8*«PP^)/2: 
x2*(  l024-weight_8*lower)/2: 
liiiewk<ih(2); 

nii<Jinax(«rray.l-max.lnuii.upper, lower): 

loopi(upper){ 

kwpjllower)  { 

set.rolor(iniuc,miii,*rrfty(i][i]); 

dr»wH(x2+weighi.8*j.y+weigbt.8*3/4. 

xl-fweigiitjs*i.other*8priiig); 

)} 


colorb«r(  1024— 2M,y+80.niax.inin): 

) 

Rndmaxl  Mrrity.inax.min  .otiU.iiw) 

float  MT»yQ(8i*e].*inM.*iniii; 
iut  oula.iiM; 

(  iut  ij.k; 

♦min  =  8rr»y[01(0]; 

•max  =  arr8y(0](0): 

Ioot>ii(out8.iiia){  if  (array(i][j)<  *11011)  *niin=arTay[i)[i): 

if  (ai:ay(il{j]>  *max)  •iiiRX=arTay[i][j); 


300 

fiuduuix 


3in 
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flndinax-duplagr.count(dwplay.c) 


) 

) 

dispUy.ccmto  lUsplay.coiiut 

{  fiuMt  oirx=45.*video,oTy=tf20.*vid«o; 

no 

roloH  1000): 

rerinoffx.offy— 3.,oirx+)10.,offy+  Id.); 
foMYELLOW): 

«ritejiit(  ( iut  )o<Tx.(  iut  )offy."  "  .count ):) 
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DO.TEST(te8t.c) 


#incliiclH  "dcflaitioM.k" 

#inclnde  ''iiat.k'* 

fioat  hiiiioo'[nH: 

axiom  wt  oii(|>iit.Jnpnt.bi<)r.onr; 

oxtem  ini  right.good.fxainplanM>nple,tc8t.rxMiv.tcflt; 

extern  atmct  nminU.DH  net; 

CIIECK.EBRORS(y.doft) 
float  vQ.iioOQ: 

{  iiit  ij: 
float  Frror(Mie]: 
iut  fomet.riglitsO; 
iut  raneet^oodsO: 

loopi< output)  {  errar(i]  =  -  y(ij; 

if  (efforp]  <  0)  error(i)  =  -ertorp); 
if  (morjij  <  O.-'i)  cofKft.right++; 
if  (errofp)  <  0.2)  correct  joo«J++; 

) 

if  (forteft.riglit  ==  (output))  right++: 
if  (correct ^oo<l  s=  (output))  good++: 

) 

float  OO.CIICCKO 
{  iat  ij.tright.tgood; 

triglit  =  right:tgood  s  good; 
riglit  s0;goo<l=0; ; 
loopi(exainplan)( 

MAKF.INF('T(ne*.inp,not.yo«it.i); 
FEEO.FORWAHIWfllEfKM ); 
CllCCK.EHRORS(iiH.y|i,iiet.yout); 

) 

ng)it=tright;goodstright; 

tent  s  (float)right/(float)examplan  *  100.; 

rettuu  test;} 

FX).TF,<5T() 

{  iut  ij; 

riglit=0;good=0;; 

loopi(exain.test){ 

MAKE.INPi'T(uet.inp.wt  yout.i+examplars); 
FF,F.».FOBWARD(<  HEt  KIT); 

<  'HEfK  EHROHS(uet.yp.net.yout); 

) 

printf("Ts#t: Eight  X3.7i  Good:  XS.W  \n". 

(float  )right/(float)exain.test*  100.. 
(float  )goo<l/(float)exaiii.test*  100.); 

riglit=0:good=0; ) 


CHECK.ERRORS 

H 


r 


DO.TEST 

40 


r 
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iiiaui(general.c ) 


* 

•  DATE: 

•  VERSION: 

• 

*  NAME:  RfM  Ntunl  Ntt-Main  Loop 

*  MODVLE  NUMBER:  1.0 

*  DESCRIPTION:  Blind  ■<<  ir.Mm  loop 

*  ALGORITHM:  Nont  in 

*  PASSED  VARIABLES:  Nont 

*  RETURNS:  Nont 

*  GLOBAL  VARIABLES  USED:  Nont 

*  GLOBAL  VARIABLES  CHANGED:  Nont 

*  FILES  READ:  Nont 

*  FILES  WRITTEN:  Nont 

*  HARDWARE  INPUT:  Nont 

*  HARDWARE  OUTPUT:  Nont 

*  MODULES  CALLED:  Nont 

*  CALLING  MODULES:  Nont  m 

« 

*  AUTHOR:  Grt  for  1  L.  Ttrr 

*  HISTORY: 

#incln<le 

#iucla<le  "dcflaitieaa.h" 

#iiichul«  <sUlio.h> 

■tract  nrural.ori  net: 

extoru  int  nienu();  an 

iut  coiinls0.rigbt.goo<i.gueas,test,di8play  =  500^bow^weigiit«=:500: 

mctmi  int  exain.test: 

iltMit  iihoodf: 

extom  int  fant: 

extern  FILE  *bi\ 

tnaiaO  UUtiu 

{  liide.iero  =  40  ; 
hide.one  =  20; 

hiile.tiro  =  13;  ao 

nhoodf  =  (float  )liide.xero; 

INITIALIZEO; 

DISPLAY.NETO. 
wUile  (count  <  100000)  { 

clierli.inonae( ); 

MAKEJNPnT(nei.iiip,iiet.<k)ft,— 1); 

FEED.FOmVARDO: 

( 'll  RC'K.EB  RORS(  net  .outp.iiet  .doR );  sn 

BAt'K.PROPO: 

/•  irati(I00){ptrlttAO: 
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iiiain(g«neral.c) 


FIXUPO:  }  •/ 

«r»it(  display  )SUOW  ( ); 
wait(8l»ow.w»ights)DlSPLAY.N  ET( ); 
wait( display)  {  calrulat^.erroK ): 

ifl^aiii.t^stlsO)  DOjrESTI): 

niake_graph( 500.  .560. ): 

} 

iftfast  *=1|  DISPLAY.NETO; 

fount+-H 

display  .count! ): 

} 

fclospirst): 

savc.amglitsO'- 

) 
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(defluitious.h) 


* 

*  DATE;  11  ..«j  19SS 

*  VERSlOU:i.l 

* 

*  SAME:  Dtfintiiont.h 

*  MODULE  S UMBER :.1.1 

*  DESCRIPTION: 

*  ALGORITHM:  Sour 

*  PASSED  VARIABLES:  Nont 

*  RETURNS:  Nont 

*  GLOBAL  VARIABLES  USED:  None 

*  GLOBAL  VARIABLES  CHANGED:  Nont 

*  FILES  READ:  Nont 

*  FILES  WRITTEN:  Nont 

*  HARDWARE  INPUT:  Nont 

*  HARDWARE  OUTPUT:  Nont 

*  MODULES  CALLED:  Nont 

*  CALLING  MODULES:  Nont 

* 

*  AUTHOR.  Grtforf  L.  Tarr 

*  HISTORY: 


#(ioau«  GRAPinrs  true 
TABLE  513 

loo()i(ON)  for<i®0;i<ON:i++) 

#(ldiu«  loo))j(ON)  for(j=0a<ON-j++) 

#d'4iu«  loopk(ON)  for(k=0;k<ON;k++) 

#d*’flue  loopij(ONF,TWO)  for(i=0;i<ONE;i++)for(j«Oj<T\VO-j++) 

IIARD.ON  RED 
#il<«ue  INDETKRM  GREEN 
#duau«  UARD.OFF  BLUE 

#if  FALSE 
#dcfUie  TRUE  t 
FALSE  0 
oiilpiii  1 
#dcfiue  input  22 
#dcdiue  liidv.on«  20 
#dnfiiio  hide.tno  2t) 

#<mdif 

#d<diti«  lin^  printfCVn") 

#dtdiiM  dmply  lOU 

#dt!flue  tiatafilv  AdataVnek.data" 

#dvflu«  spring  150 
#d<dluo  nriglit.s  20 
#iUdiu«  waiMA)  if((connt%( A))s=0) 

#d(3liue  squarHA)  ((A)*(A)) 


.»2;07  Nov  M  tSSS 


Jn 


y 
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NETA  P.3 
fiid.koli  10000 


(defluitions.li) 
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(nel.h) 

size  100 

struct  neursl.iiet  { 

float  oiitp{sizp]; 
float  dof)[size); 
fltMit  w2(size](size]: 
float  aw2[size](size]', 
float  t2(sizel; 
float  oiilp.iniisk[size]: 

float  y2(size]; 
float  y2jna»k[8ize]: 
float  y2.dl[size]: 

float  wl[size](size]: 
float  air  I  [size] [size]; 
float  tlfsizej; 

float  ylfsize]; 
float  y  ljii*sk[sizel; 
float  y  l.dl.(size]; 


float  w0{8izej[size]; 
float  air0(8ize](aize]; 
float  tOfsize); 

float  wk(8ize][8ize); 
float  awk[8ize](size]; 
float  tk[8iz<<]: 

float  k  I  [size]: 
float  kl.uia8k[8ize]: 
float  kl.dl[size]: 
float  kl. roll  (size); 
iut  kl.rlaiiii(8izej; 
float  iiip(8izej; 
float  iiip.iiiaskfaizej: 

iut  r|aSf,Jesl[sizej: 
iut  class .rouiU[sizej: 

iut  inpiU.liiile.zero.liide.oiie.liide  two.oulpiit: 
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INrnALIZE(iiutiali»!.c) 


*  DATE:  3  Anfa-)/  iJWi 

*  yEES/OX 

*  yA3fE:  InHialhttltnn  Modalr 

*  MODULE  NUMBER:  1* 

*  DESCRIPTION:  ItnUati:ai.on  of  iht  iuplof  konlwon. 

*  onf  inpot  tf.it  doto.  and  net  data  strnctair. 

*  ALGORITHM:  Nont 

*  PASSED  VARIABLES.  Noat 

*  RETURNS:  Near 

*  GLOBAL  VARIABLES  USED:  Nft 

*  GLOBAL  VARIABLES  CHANGED:  Nont 

*  FILES  READ:  Nont 

*  FILES  WRITTEN:  Nont 

*  HARDWARE  INPUT:  Nont 

*  HARDWARE  OUTPUT:  Nont 

*  MODULES  CALLED:  Nont 

*  CALLING  MODULES:  Nont 

*  t^THOR:  Grtforp  L.  Tarr 

*  HISTORY: 


#iiiclnde  "daflaltlou.h" 
#iuclu(le''gl..)i" 
#iuclud«'’dia«le«  •ti*' 

#iuclndn  <liiiM*.li> 
#iucliide  <stdio.h> 
#uicliid«  <sigiial.h> 

#iuchid«  "aat.h" 

extmi  struct  ncuraljict  net; 

float  raiidond ); 

#d«flue  TABLE  512 

extern  iut  nieiiuO; 

extern  float  roiucience; 

INITIALIZE! ) 


V>«l|NKMNT.i!ieiiu ): 
iiiit.urreenl ): 


:J:0S  Nan  30  ;9.M 


30 


V 


30 


40 


INITIALIZE 


P9ff  1  of  inthnhze.c 


I 

I® 


INITIALIZE-init.net(initiali*e.c) 


} 


nmiu( ); 


{ 


O 


iut  ij: 

ii«*l.w2[i|[j]  =  ranrloni<); 
loo|>i(aixe)  ii<ft.t2[i)  =  ranJoniO: 

loopijfsizr^izr) 

ii<-l.wl(i][i]  =  raiuloinO; 
loopi(sm)ii«l.tl[i]  a  rantlomO; 

loopij(«»r^)zr) 

iifi.irO{i][j]  =  raii<ioni(): 
loo|>i(8is»)  uel.lOfi]  a  raudomO; 

loopijiMZrjizr) 

ucl.wk[i][j]  a  0.0: 

loopdaiie)  iK‘l.lk{i]  a  randomO; 

loopi  (ii»)nrt.kl.cou(ij  a  0.0; 

loopi(aiso)  ii«l.kl.claini(i]  a  —1; 

loopi(aitr)  n<*t.y2.maak(i]  a  TFU'E; 
loopi(aiz«)  nrt.yljHaakii]  a  THUK; 
loopi(ai**)  nvt.kl.niaak(ij  a  TRI'fC; 
i4»ipi(»ii«)  a«-t.inp.niaak(i)  a  TRUR; 
loopi(aiz«)  iirt.oul.p.njaak(i)  a  TRUE; 


mit.uot 


i«<i 


ao 


} 

Bout  random  () 

{  flout  x; 
iut  y; 

y=(raml()  %  100)  +  (rand()  %  100)  +  (raiid()  %  100); 

xa((Bo«t)>7300.0-0.5); 

rotiuru  x; 


C 
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FEEO.FORWAIlD(feedforward.c) 


1 


DATE:  II  Sfpt  1934 
VERSION: 

yAME:  Fttdforwani 
MOOI'LE  yUMBEIt:}.4 

DESCRIPTION:  Pmndt»  fonnari  propagalton  of  thr  input  tignul. 
ALGORITHM:  fudfonrord 
PASSED  VARIABLES:  Non* 

RETVRNS:  Non* 

GLOBAL  VARIABLES  USED:  Non* 

GLOBAL  VARIABLES  CHANGED:  Non* 

FILES  READ:  Non* 

FILES  WRITTEN:  Non* 

HARDWARE  INPUT:  Non* 

HARDWARE  OUTPUT:  Non* 

MODULES  CALLED:  Non* 

CALLING  MODULES:  Non* 


AUTHOR:  Grtforg  L.  Tnrr 
HISTORY: 


#inel)id«  "d^flaltioaa.h" 
#iuclutle 

#iuclud«!  <ni»lli.b> 

mcturn  iut  nijiiinnini,nliood.ccyp«: 

ext«‘ni  flout  alplia; 

cxtinn  struct  ncuraijirt  net; 

«(XtsTu  iut  con  III  ; 

float  rsIryO; 

float  ralrkO; 

iut  krlMs{8Uc]; 

float  niin.disl: 

float  roiisrii*nrr; 

FEED.FORWAHDO 


(  iut  ij, winner; 


loopi(iiipiit)  if  Oirl.inpjiiaskfi}  ==  0.0)  net.iDp(i]=0.0: 

/•  h'ohonfn  Lnjir  */ 
loojii(liiile.*ero) 

{nel.kl[i]=  r»lck(  iiet.inp.net. wk. net. lk.&;i.nipul);] 
Hn<ljiiiii.n<HM^''v>>iiirr);  niiniiinim  =  winner. 


/*  Rffular  f**dforward  mth  mank  for  nod*  trark*r  */ 
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FEED.FORWAIlD(feedforward.c) 


loopi(Iude.oii^) 

caicy(net.k  .nrt  w0,iiM.t0.i;i,hide.xrro);} 
loopitliidr.oiw)  if  (n<^t.yLniaak(i]  =-=  0.0)  nel.y  l[i]=0.0; 


loot>i(oulpul) 

{ net .outp(i]=ciilr y(  iM't .y  I  .net . w  1  .net . 1 1 .1: i . Ii i< le_one ) ; ) 


W) 


#if  FALSE 

loopij(liide.tero,inpnt) 

(  if(uel.iiip.iiiMkli]==1.0)  iiet.awk[i][i]  =  uet.irk(i1p]  *  nel.iup[j];} 
loopij(  hide.one.hide.zero) 

{  ilinet.kl.nuuiky]==1.0)  uet.aw0(i][)]  =  De{.wO[i][i]  *  iiet.klp];}  to 

loopiji  hide.lwo.liitle.one) 

(  il)net.yl.niMk{j]a=l.C)  nei.awl(i}[|]  =  Det.trl[i)[j]  *  aet.yl[j];} 
loopij(ontpnt,hide.twu) 

{  iitnet.y2.niaak[i]a=1.0)  uet.«n'2(i][i]  =  net.w2[i][j]  *  uet.y2[)]:) 

#eu<lif 

} 

fltmt  fixyO; 

float  esiey(x.w,theta.index,loaer)  an 

float  xO,wQ[«iie].thet»{]; 
tut  *in<lex: 
iut  lower; 

{  iut  i.k.nuniber; 
float  y: 

y  =  0.0; 

nninher  =  *index; 

ioopi(lower){  y  =  y  +  x(ij  *  w(miniber][t];} 

y  a  y  —  theta(nnniber];  !>o 

rotiuru  fixy(y,l.0);} 


float  fixy(y.hardnew) 
float  v.liardneM; 

{ 

rrtivu(  l.0/(1.0  +  (float)exp(— (<l<Nible)(lmrdneoR  *  >■)))): 

) 

float  ealek(x,w.lheta.index.lower)  loo 

float  x(].w[][size],tliet«Q; 
iut  *index; 
iut  lower: 

{  iut  i.k.iiuinber; 
float  y: 
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FEED.FORWARD-andxiode(feedforward.c) 


y  =  0.0; 

.  ■iiiinlM’r  =  *iiulw. 

loo|ii(  lower)  {  v  =  y  +  s(|imre(x(i]  —  w(iiuiulietl(ij); 

/*  printfCrfi]  %3.3f  w[i}  %3.if  «fr  %3.3f  g  %3.3f  \», 

T[i].w[namitr][i].sg»trt(r[i]-v[nami€rJ{t]),i):*/] 
retoru  (flont)sqrt(((loubte)y);} 

flout  vigileiire  =  0.1; 

fi  IK  1 .11  ti  h.imxM  will  Her ) 
iut  *wiiiiier; 

{  iut  i; 

floMt  iiiin.bi.ii.<lMtanre; 
iut  iiiiu.iiode; 
min  =  100000000.0; 
mill.lKvle  =  0; 

looiii(hi*le_*ero)(  bi  =  iiet.kl.cou(i]/(flo«t)rouiit  —  1.0/(flo«t)hi(le.xero; 

iflbi>  ronnrienre)  iflatpiia  >  0.0)  eontinne: 

iftiiet.kl[i]  <  vinileiiee)  {  niin.iiorie  =  i;  break;) 
il)iiei.kl[i]  >  mill)  coutiune; 
min  =  net.kl(i];  niin.iKxie  =  i:} 


net.kl.con(nun.node]  +=  1.0; 
Det.kl.clnini{min.node)  =  ctype; 
‘winner  =  niin.uode; 
krluB[niin.no<le]srtype; 
niin.i'Mt  =  min;  • 


loo|N(liide.*ero}( 

net.kl[i]  =  1.0/(010  +  net.klfi]);) 
net.kljmin  node]  *=  2.0; 

) 


niiiliioilHu.ys) 
iut  xs.ys; 

{iut  i.lnyer.llie.node.temii; 

layer  =  (y«/»|King): 
lliejiode  =  (iut)(x«  /  weight.*); 
temp  =  ( 1024/weiglil.*): 
f  lear.»freeii( ); 

•witcbl layer)  ( 

caae  0:  llie.no»le  -=  ( temp— input )/2: 

if(net.iiip.maak(tlie.no<le]  ==  0.0) 


fiull.uliu.uoile 


l?0 


130 


HO 


fiutluodc 

mn 
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Poft  3  of  fer8/orw»nl.  r 


flndno<le(fecdforward.c) 


n«t.inp.mMk(tb«jBo<i«]  =  l.O; 

else 

oet.inp.masklthe.Qode}  =  0.0: 
brenk: 

cese  1:  the^ode  — s 

if|net.kljiisak[the.iK>«lc]  ==  0.0) 

{  net.kl.niask(the.iio<l<>]  =  1.0; 
loopi(hide.oiie)  net.tOp)  +*  net. «w0(i)(tlie  node];  ) 
else 

I  {  iiet.kl.niask(the.nodej  =  0.0; 

loopi(liide.ooe)  net.tOfi)  -=  net.swO(i)[the.iiode];  } 


c«se  2:  tlie.ntxle  -=  (tenip-hide.one)/2: 
iftiiet.yl.niask(tbejiode]  ==  o.O) 

{iiet.y|.niMk[tlie.no«l<‘]  =  l.O; 

loopi(bide.two)  uet.tlpj  +=  tiet.*el(iI(the.iKKleJ;  ) 

else 

{  net.yl,nuMk(the.iiode]  =  0.0; 

loopi(hide.t*o)  nei.ll(ij  -=  nei.swl(i](the.nodeJ;  } 


case  3:  the.node  -s  (tenip— bide.two)/2: 


if(net.y2.nuMk[llie.iiode]  ss=  0.0) 

{  net.y2jiMsk(tlie.node}  =  1.0; 
else 

{  “ci.y2.inaskitSie.iiode)  =  0.0; 

break; 

) 

?oo 
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BACK.PROP(b«ckprop.c) 


DATE:  28  J,lg  1983  • 

VERSION:  • 

• 

NAME:  Btrkprop  * 

MODULE  NUMBER:  * 

DESCRIPTION:  Using  output,  adjusts  mgkts  irduct  tmr.  * 

.■\LOORITRM:  llVrio*  Muttilagtr  Ptmptron  Baripnpufution.  *  la 

P.XSSED  VARIABLES:  Aon.  • 

RETURNS:  Nont  * 

GLOBAL  Variables  used:  vtrurs  * 

GLOBAL  variables  CHANGED:  Wtifki  Vtctors  * 

FILES  READ:  None  * 

FILES  WRITTEN:  Nont  • 

HARDWARE  INPt’T:  Nont  • 

HARDWARE  OUTPUT:  Nont  * 

MODULES  CALLED:  Nont  • 

CALLING  MODULES:  Main  Loop  *  jo 

• 

.AUTHOR:  Grtforg  L.  Tarr  * 

HISTORY:  • 

* 


#<l«5Ane  TUETA  TRUE 

#ineln«l«  so 

#iurhi<l«  "nat.h** 
extern  ttmct  npurM.net  net; 
extern  iut  count,(li«|i|»y; 
floet  neta  =  0.3; 
flout  alpha  =  0.3; 
floet  <<elx(); 
floet  dely(); 

iut  miiiinium.nliood; 
flttet  iihoodf; 

BACK.PROPO 
{iut  i j.k.ii.nui>i; 
float  mill; 

floftk  (iel3(9isp|.dcl2(]iiz«].d<fl  1  (mspj.dolOffluej; 


40 

BACK.PR0P 


if  (ronnt  >  etui.koli’  { 
!*  output  */ 


20:19  Ort  4  1988 


Puff  I  of  kackprop.r 


BACK.PROP(backprop.c) 


loopj(outpiil)  {  =  dely(net.outp[jl,i»et.doft[j)); 

loopi(hidf.two){ 

net.w2Dlti)  +=  (iirta  *  delStj)  *  i»t.y2(i])  *  iiet.y2.mMkli): 

)) 


/•  SrcoiiJ  Hiidtn  */ 

loopj(liid*.two) 

{  dfl2[j]=  d«-lx(Drt.y2li].<W.rnrt.w2J.outpiit): 
ix^.tlQ]  +=  *  «W2[i]  ; 

toopUhidf.oiH*)  {  .  >  M 

nrt.wlCiKiJ  +=  net.  •  dH2(j]  *  net.ylpl*  nrt.yl..ii««k01: 

}} 


/•  Fini  HMtn  7 
loopjiliu^.on**) 

{  <IH  1  [il=d**lx(  nrt.y  I  li],<W2.iM*t. «r  1  J.hkJe.two); 
iHfl.toO]  +=  11^  *  delltil  ; 
loopi(hid«.irn>)  { 

nrt.wO(il(i)  +s  nrt*  *  dWl(jl  *  Bet.klpl  *  iirt.kl.ma.k(i): 

)  } 

} 

/*  imikah  i$  tkt  p»i»t  whtrr  kokontn  trminimf  i»  itmti  off  7 

iftrount  <  «i<l.koh  )  { 

/•  •ijant  mtifkkorhood  (vtrti  frw  romnU  *f 
wwl(200)(  nhoodf  =  nhoolf  *  O.tW; 

alpha  =  alpha  *  O.OS: 
colorl"): 

rwm(-l5,”5, 200.05): 

nhood  =  (iiit)ahoodf; 

writ*-  iuM 50, 80."Ialshborhood:  ".nhood); 

} 

iftnhomi  ==  0)  nhood  =  1; 
if|count<400)  nhood  *  hidp.iwo: 

/  •  fimi  min  Manrt  nod«  7 

mini  =  minimum: 

/*  nd/Ml  HfSfkkorkoad  wtifkU  */ 

for(i=  mini—  nhood:i<iiHni+nhoo«l:i++.kssiH*i||hboni(i.nhood.hid^.WTO)) 

loopj(inpnt)  nct.wk(k)lj)  +®  alpha  *  (  nrt.inpQ]—  n«H.wk(k)(j)):} 


Oct  4  19S8 


Paff  *  of  karkpntp.c 


_J 


BACK.PROP-neighbor8(backprop.c) 


1 

.KMghbo«(i.neigl..l.y«)  Ucigkbors 

iut  i.ungliJayrr; 

{  iut  x: 

.x=Uy«»T+i: 

elM  x=i  %  layer; 
retnru  x; 


1 

float  (IHy(y,(lof() 
float  y: 
iut  doft: 

{  float  delsO.O;; 

d.-!  s  y*(l-y»*((float)doft-y); 
r«taru  del; 

1 

float  tJ**lx(x,dH.w,n.uppef) 
float  x.delO,wQ[*i*ej; 
iut  n, upper; 

{  float  deUa^nim; 
iut  ij; 

•uni  =  0.0; 

loopi(up|>er)  »»nn  =  ium  +  del[i)  *  w(i)(u); 
sum  =  x*U-*)  * 
return  sum; 


} 


.>«.  W  Ort  4  IMS 


Paijr  t  of  horkfrop.  c 


showinput(show.c) 


/•••**••••*•*••• . . . . 

« 

•  DATE:  1  Otloitr  1988 

•  \EIiSIOt\:  1.0 

0 

•  .  yAME:3iow.e 

•  MODl'LE  SVMBEIt:  1.5 

•  DESCRIPTION:  Diaplai*  luieruul  valtea  m  tht  Itripori 

•  ALGORITHM.  None 

•  P.\S.SED  VARIABLES:  Nont 

•  RETURNS:  Nont 

•  GLOBAL  VARIABLES  USED:  None 

•  GLOBAL  VARIABLES  CHANGED:  Nont 

•  FILES  READ:  Nont 

•  FILES  WRITTEN:  Nome 

•  HARDWARE  INPUT:  Nome 

•  hardware  OUTPUT:  None 

•  MODULES  CALLED:  None 

•  CALLING  MODULES:  Nome 

0 

•  .AUTHOR:  Greforp  L.  Tmrr 

•  HISTORY 

0 

00,0,„0,0,.,00,0,0„00.00,00000„0..0.00.„.„0,0„„0,.,0„.,0..,0„„0f 

#iueln<l« 

#tucliicle  <st<lio.h> 

#iuchi<ki  iBitiou.h* 

extent  (tract  nrarkl.nct  net; 

extern  ittt  rount.right.guew.good.test.dnplay; 

extern  Bont  iihoodf; 

extern  float  alpha.conarieure; 

iut  10  s  1; 

iut  II  =  0: 

iut  12  s  0: 

iut  Ui  =  0: 

iut  14  =  1: 

extern  FILE  *bt; 

SIIOWO 

{  iftIO)  «liowiii|>ut(iiet.iiip)  ; 

ifl  14 )  ahowoutpul( iiel.oiitp.iiet .doit ); 

priiitn "Alpha  >  X3.2<  Coaseiaaea  X3.3f\a".alpba.roii8ririice); 

»howuodf(  iiel.k  l.rou.hi<le.xero,''Ceaaelaaca" ); 

/*  .nko«<wetflita(nrt.irk.mtl.kLfon.l.hidt  ^fre.mpal^; 

V) 

aliowiupiiMx) 
float  xQ; 

\  bit  j.i: 

P«rc«ptroB  Nod^lNa**); 

(ou|>i(in|Hil)  X(Xd)  " 


10 


20 


30 


SHOW 

40 


sUowiuput 

50 


1 
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Pnf(  /  nf  nhow.c 


ahowinput-showweight8(8how.c) 


loopi(iuput)  printfC^l-Sl  ".xlill'JineUine; 

} 

float  priglit.pgootl; 

sbowoiit  pul  ( y  .dofl )  sllOWOU  tput 

float  >  Q; 

float  JoflQ;  ■'<' 

{  iut  ij; 

float  •'TTor; 
printfCOat:"): 

loopi(output)  pnnlf("t3.2f  ",y[i]);lin«; 
printf("I)olT;"); 

loot>i(output)  printfC  X3.2f  ".do(t(i|);  line: 

priKhl  s  (float  iright/dinplny  *  100.0;  to 

pgo^  =  (flf>at)good/<li8plsy  *  100.0; 

|irmtf("C«nAt;Xd  \B&lght:X2.21  (SaMa:X3.2f  Xa” .count. 

pright.pgood); 

fprinir(lst.‘‘CoaBt:Xd  ftlght:X3.3f  <inaa«:X2.2f  ".count. 

prigbt.pgood); 

ffliMh(f8t); 

rightsO; 

goodsO; 

«n 

) 


oliownotHy.n.mnw)  sboWliOtle 

float  yO: 
iiit  n; 

rliar  menaQ; 

{  iut  ij; 

float  inax.inin;  m 

line; 

priiitf  ("Xa  ".ineoal'.line; 
ioopiln)  prinlf("X2.2f  *,y[i]);liiie; 

} 

»lio«weighla(  w.theta.layer.iipper. lower)  sltOXVTVoigllt.S 

float  wO(aize].llietaQ: 

iut  layef.upper.lower;  hki 

{  iut  j.i; 

float  iiiax.niiii: 
line: 

loo|»i(lowef )  { 

loopj(iip|>er)  I  if  |w[i][i]  <  0.0) 


li:t0  Sop  iO  198$ 


Poft  S  of  fkow.e 


lini*;} 

prinif(  "\nTli*ta«:  \n"); 
loo|»i(ii|>p<fr){  priiitf(  "Xl.ll  ".lheta(i]):} 


(inenu.c) 


* 

*  DATE:  -1  Oclohtr  1933 

*  VERSION:  2.0 

t 

*  N.-\ME:mtnn.e 

*  MODULE  NUMBER:  2.6 

*  DESCRIPTION:  Proridts  of  inlergclipt  inruu  opltoot 

*  ALGORITHM:  Noat 

*  PASSED  VARIABLES:  Nont 

*  RETURNS:  Nouf 

*  GLOBAL  VARIABlcS  USED:  Nont 

*  GLOBAL  VARIABLES  CHANGED:  Nont 

*  FILES  READ.  Noar 

*  FILES  iVRITTEN:  Nont 

*  HARDWARE  INPUT:  None 

*  HARDWARE  OUTPUT:  Nont 

*  MODULES  CALLED:  Nont 

*  CALLING  MODULES:  Nont 

« 

*  .AUTHOR:  Grtqorp  L.  Ton- 

*  HISTORY: 

* 

axteru  iu*  dm.rlMi^Miipl^.exani.tnt; 
flxtftru  floMt  iiliocxlf: 
exteru  float  add.noiar .alpha: 
extant  tut  artiv^how.neighta.dmplay' 
extern  float  Ihrceliold.adu.noiw.coiiscieiKe: 

#inf'liule  <«ign«l.h> 

#aiehide  <stdio.h> 

#iuchi<ie  <gl.h> 

#iuclu<le  ./vacker/deflaltiOBa.h" 

#iuclnde  "aet.li" 

extern  iut  n<iiitakn; 

exttvu  atnict  imirat.net  net; 

exteru  iut  rouat.right.good.guess.test; 

exteru  iut  hide.one.hide.zero.liide.two.iiiput.output: 

iut  iitennO 

{  rJiar  seii’rt.filen»nie(20l: 

iut  uodenuiiil>er.sel«'tor=TRl'E; 


systmi(  "clear"); 

priiitf  ("  Xaaa  \n"); 

priiitf  ("  1}  laitialize  Systeu");liiip: 

priiilf  ("  2)  Sara  Weighta"):liiir; 


22:10  Nop  :i0  1933 


Pojt  I  of 


(menu.c) 


priiilf  ("  3)  K««d  Uaights 

|>rintr  (**  4)  Teggla  4et/V«i^ta“):linr; 

priiUf  C  6)  Add  loia«"):Uii^: 

printf  ("  0)  Diaplay  lBtarTala'');line: 

prinlf  ("  7)  Toggla  Errera 

prinir  ("  8)  law  lat  Tepelegy'*);liue; 

priiitf  ("  a)  niT  \a");  fo 

prinir  ("  SELECnOI:  7  "); 

ffliwh(atdin); 
scirrt  =  grlrhari); 
awitcla(wlrrt)  { 

CMa«  'a'; 

g«it( ): 

<rxil(0); 

bnwk; 

CM«  *1*;  init..daU(); 

priiilfifEBtar  alpha, eoaaeiaaea,flrat.aaeead:\a'*):  ro 

/•  a/pAa  =  t.(i:com»fttnn  s  O.O-i.-htde^ztro  =  tO: 
kide,ome  =  S:  *! 

sfanli|**Xi  Xt  Xd  Xd‘'.Acalpba,l:ronacianre,A;iiidr.ZFro.A;bide.one); 


priulf  (”\aX3.3f  X3.3f  Xd  Xd\a*'.alplia.coiiacieiice. 

liide^ro,hide.on«);  ao 

ioil.D«l( ); 

;bt«ak: 

caa«  '3*:  wril«.w«ghta  (): 

;brtHtk; 

cMe  *3';  priiitr("\aBatar  FllaaaM:  \a  T  '); 
arann”Xa".fil«iaiiie); 
rmd.waighta  (flkiiainr); 
priiitVCNa  Xa  Zaatallad\»\B",fl)«iiainF); 

break;  no 

cnae  '4';  artiv  =  TRCE; 

break; 


caae  'S’:  printDCloa  each  aelaa:  ?"); 
aeanf("Xf".i:add.noiar); 

break; 

caae  'S’;  prinlfCNaCoaBt  beteeea  Scraea  Opdata  ?\a");  ion 

sranfl  "Xd" ,  A:ahow.w«ig|it8 ) ; 
prinin''\BCeaBt  beteeea  teata  TNa”); 
sranR  ‘'Xd”  .Ardiaplay ); 


break; 

caae  '7':  iiiialakm  s  TRl'R; 
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read.weight8(  menu.c ) 


breiik: 

enae  ’8’; 

l>riutf("I««  latwotk  TopologyXa" ); 
l>rintf{’*lwib«r  ia  Kokoaaa  Lay«r\a“); 
scaufl  "Xd".i:hi<l<“.»^ro); 
printf(**8wib«  la  Flrat  Uy»r\a"): 
scan  f(  "tA’'  hidc.oiie ) ; 
priutrC’laabar  la  Saeoad  LayarVa"); 
sc»iiIl"Xd".4ibide.iwo); 
initjict( ); 

nlioodf  =  (float  )hide.*cfo: 
alpha  s  0.!i; 

nboodf  =  ( float  Ihidc.wro; 

priiKfl "alpha; Xl.3f  aho«d:X.3.21  ",8lpha.nhoodf): 
break; 


sigiiaKSKj  INT.nieiiu ): } 

real I. weights  (filename) 
rJiar  filenanii’Q; 
j  FILE  *rp; 

float  x; 
uit  ij.k; 

fp  =  fopen  (filenani<'."r“): 
if  (fpssNl'LL) 

{|irintf|"\a  aaafila  Error***  \a");retnni;} 
fmrnnf  Itn  "Xd  Xd  Xd  Xd  Xd  ", 

ri)«tput.4:bide.two.iliide.one.i;hide.xero.kmput); 


rpacl.woights 


IM 


loo|>ij(o««tpnt,liide.two){ 

bcaiif(r|>."Xl".kx); 
iH*l.w2[i][j]s:x;} 
loepij(  hide.two.hide.on«){ 
fscauf(fp."Xt".fcx); 
iiet.wl(i]0)=x;) 

loopij(hi<l«.oiie,hide.Kro){ 
fscanf(rp."Xf".4:x): 
wt.w0(ij2]=x:) 
loopiji  hide.iefo.input )  { 
facai.f(fp."Xf".kx); 
net.wk{i](ilsx:} 

C»eaiif(fp,"XI  XI  ".fcnIiocKif.k-alpli*); 

loopi(o«tpu()  {facanfirp."Xl".net.t2+i);) 
loopilliide.lwol  {f*c*iif(rp."X*".iHrl.H+i);) 
Inopi(hidejone)  (t;caiirifp."Xt".net.tO+i);J 
facaiifl  fp."Xd"  .&<  ounl ); 

looin(lihK»*ro)  {facaiir(fp."Xi".iirt  y2.m*ak+i):} 
loopi(liide.one)  (fsc*nf(fp."Xf".n.-t.yl.nia*k+i);} 
lo*>)»H inpnt )  {farann  fp ."XI "  .net .inp.inaak+i ); ) 
rrlo*F(rp);) 


a 


j 


j 


ri 


3 


??./d  IC  I9t3 


Paft  i  V  mtna.e 


•w 


\ 


% 


D 


C 


c 
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read.weight8-«ive.weighta(menu.c) 


writ<f.wighU  () 


{  FIIX  ‘rp; 
hit  ij.k: 

rlMir  fil»*n»iw(20]'. 
prini(i("\ii£Bt«r  “)• 

fp  a  fop^n  (filtfimniif. 
fprtnrf  (fp.-W  Xa  Xd  td  Xd 

output  .liiUr.iwo.hiUe^one.liKMr.tefo.iiipiH). 

fpfintf(rp.-Xi  ".nM.w2(i)ljl.ij)-.)  fptmtflfp-  >•> 
loopiji  hidO^ohi'^'.on^  I 
fprintf(rp.-Xl 
loopij(  hi*!** 

fprintRfp.-Xf  “  iirt 


loopii(  input ) 

fprii.tf(fp."Xi  -.n«<.wk(ilD).ij): 
fpfintf(fp."Xf  Xf  ".nhoodr.iiJpha>; 


V'Q 

TtTite.wpigiits 


I  TO 


1«> 


loopi(ontput)  {fpffntf[rp,"Xi 

loopi(lii<lf.lwo(  {fprii.inrp."«  I  );} 

loopi(hiJe.on*)  {fprintf(rp.-Xf  \a".iHf».toW).) 
fpfiiitf(fp  "Xd  \a-,fo.Hil): 

loopi(l.i<l*.two)  Uprint«rp.«Xl  \mMH-t.y2.mMk  I  : 
Joopidiide.onf)  <rp«n‘f<n'."W  \*MH-t.yJ.nn»k(.));) 
>.oopi(i«puH  {fprt.itf(fp -XI  Va-.n^-t  inp  maakli));} 
frloii^fp»;priiitn-\*  ¥•!»»«•  StoradX*"); 

) 


Mvr.wrighta  () 

j  FIIX  ’fp; 
hit  ij.k: 

clrnr  filmani'^201'. 
fp  =  fo|Hrn  (-kr^ld,B*t. 
fpriiiif  (fp,-X4  Xd  Xd  Xd  Xd 

oiitpHt.liid^.two.hidr.oiw.hid^.two.inpiil); 

loopii(oiitpitt.l!i<lr.t»''o)(  1  V  .  1 

fpriiiltl|rp.-«  -.nrt.w2(il[jl.i4)-4  fprintflfp.-X*-).} 

Ioopu(  hiiMw.li'd*.*"'*  f 

fpriittfirp.-i«  -.urt  wiplUlaj); 
loopij(  hitk.w  .liid^.wro) 

fprii.tfl|rp.-W  -.iHd.frO|i)lj).i4): 

loO|*ij(  hi<l«*,*«w.inp«t ) 

fprintnrp.-Xi  -.iwt.«rkli)ljl.ij); 

fprintHfp.-X<  H  ".iiliopdf.«lpb»); 


iw 


save.wciglits 


3|0 


C 
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8ave.weigfat8~write.int(inenu.c) 


oopi(o«ti«it)  {fprintf(fp.-Xj 
loopi(liide.two)  Ifprintfirp.-XJ 
loopi(hi.lf«.on»)  {fpriii(r(fp."«  \tt".iiet.tOfi|):l 
fptinir(fp,"1Cd  \n",rouiii); 

loopi(hu!.*.two)  {fprint.f(rp.-Xf  \a-.iiet.v2  niMkril)  ) 
oopi(hid».on^)  {fprintf(fp.«tt  Varitrt.yl.aiMkfi));} 
loopi(iiipiit)  {rpriiitr(rp.'*%i  u-.iw  •inp  niasklt]):} 
frlos^fp).prin»(l"\a  Haighta  StoradSa-): 


?30 


writ^.striiig(x.y.I.title) 

iu»  x.y.i:  wnte.striug 

ciMur  titl«>[20): 

I  cImit  nMinltor[20|: 

I  =  (u>t)((lloiit)l/vid*o);  »o 

folorftj); 

rwtfi(x—5,y-.').x+l.y+15): 

roloffd); 
lilH^dlh(  1 ); 

K<ti(x-5.y-5jt+l.y+|5); 

ciiiov2i(x.>-2); 

rliMatr(lille); 

) 

writ.-.float(x.y.J.(itt»,n,«.roteri  .  ^ 

iMt  x.y.i;  ’  'tVTite.floaf. 

cltar  iill<'{20]; 
flont  n; 

•Mt  a.rokw; 

I  rjh«r  nnnil>«ir[20]; 

I  —  (iut|((flont)l/vH]m); 

»printf(  ii<inih«T,’^ .  ); 

cok)r(«_rol<jf); 

nTtfi(x— S.y-S.x+l.y+lS): 

foloHYFLLOW): 

linewidtli(l); 

••it»v2i(x,y-2); 

rimratr(utinilM>r);  ) 

wtit«?.ini(x.y.tiil^.ft) 

ui»  x.y:  Write.iut 

clutt  t-t|»(20l;  '  vm 

iut  ft: 

j  rJMT  iiuiiihrr{2t)]; 


iJ:)0  ,Vor  iO  ]9SS 


P»fr  -!  of  mrno.f 


write.int-write.aD.int(menu.c) 


coior(  VELLOWj; 

S|>ruitf(  iiunib«*t."Xd".ft ); 
cniov‘2i(x.y); 
color  (BLl'E); 
rti«rsir<  title): 

cliamtrliiuniber):  }  iro 


vrrile.aii.inti  x.y.fl- )  WritP.aujut 

iut  x.y: 
iut  ft: 

{  cImt  niinilM>r[30): 

5|iriiitf(iiun)bHf.**Hd".rt ): 
ciiiov2i(x.y); 

cnarstrl  number):  }  2su 


*i:IO  A’nr  iO  19.1S 


Ptft  6  of  lUfnt.c 


DISPLAY,NET(dwplay.c) 


• 

•  D,\TE:lO  August  1988 

•  \ERSlOS: 

» 

•  yAME:  DISPLAY. SET 

•  MODl’LE  SVMBER:  1.7 

•  DESCRIPTION:  Displapa  intrrnai  vainta  in  a  graphtc  format 

•  ALGORITHM:  Soar 

•  P.ASSED  V.ARIABLES:  Noot 

•  RETVRNS:  Son, 

•  GLOBAL  VARIABLES  CSED:  Noat 

•  GLOBAL  VARIABLES  CHANGED:  None 

•  FILES  READ:  Nont 

•  FILES  WRITTEN:  Noae 

•  BARDWARE  INPCT:  Noat 

•  HARDWARE  OUTPUT:  Non, 

•  MODULES  CALLED:  Noat 

•  CALLING  MODULES:  Noiu 

• 

*  .AUTHOR:  Grrforf  L.  Tarr 

•  HISTORY: 


#iuclKd«  "snt.h" 

#iuchul«  “dnliaitleu.k*' 

#iucltt<le  "gl.k" 

extern  iut  fo«int.ri$ht.$ood.t<(«(.mininium.«ty|>e; 

extern  ntract  netirnl.nrt  net; 

extiim  krlamQ; 

extern  flont  prisht.|>go<Hl: 

int  Mtiv  =  FALSE: 

DISPLAY.NETO 
{  iut  ij.k; 

clier  rk»{  l]: 
ilo.ncMeiil ); 

write  ji|.ring(4X0,iK)0,2-'iQ.”Value  6««aa  Rlflit'*); 

rrrile.strittg(-480.M0.ir,0.*'Danlrad  Ontpat"); 
trrite.»tring(I0.~30.300.”l;brid  Fropagatlos  lataork"): 


if  (!  wt\v)  { 

plotnmle  (O.nH.inii.net.inp.ninnk.net.kl.iiiiiiik.net.W’k.iiipiit.liiile.zero): 
ploinoih'k  ( i.net.kl.uet.kljiinak.iM’l.y l.mmk.net.wO.hule.zero.hule.one); 
ploliiode  (2,net.yl  .iiH.yl.niiwk.iiel. outpjnaak.net. wt.hide,one .output); 
write.niringi  lt)t).220.l20.'’tfaigkta”);| 


elae  { 
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VI 


V 


3U 


DISPLAY.NET 


4n 


Pdff  /  of 


DISPLAY.NET— plot  noded  ( display  .c  ) 


plo(iKKl«k  ( 1  .ii«*l.k  1  .iM'l.k l.r»a«k.n«ft.y l.mMk.iiet.a«0,lii<l«*.i<fro.liul«?.oii^); 
plotiicKltf  (2,ii«*t.yt  .lift. yl.in«sk,n/'*.oifJp_iJia»k,in't. aw l.hide.one .output); 
writf.iitriiigi  1 00,’.’20. 120."*cti»atioii" );  I 

plotnotlriC-l.ii^t.ontp.oiitpiit); 
plotiio<l<^!(3.U.n<fi  .doR'.output); 
p|otiio<l<?tl(3.100.n«t.ilori.outpiit ); 
ploliiot  It'd)  3.200. net  .doft. output): 
plotnot^r)  0.7.0.1U-; .  k  1  .Jiidf.wro): 
plol.<>tal.s( ); 

flwk  iiMuwi): 

1 


p(otstaU<( ) 

{ 


wnt«*.floa((  10..320.1.30."!light :  ".priglil .  (nflfl); 
wrile_lloal(  180.320. 130. ‘'CMat:'*.pgoo<l.i000):} 


float  tUmliold  —  0.0: 

plotiKKl«i(  x.nodv.low^ ) 
iut  x: 

float  uod«^; 
iut  lowf'r; 

{  iut  i3.k.y..x2.y2: 


y=sp<-«ng*.x+3fl; 

x=!(  1024— w»ight.**lowM)/2; 

loopi(low<>r)  { 

»'l.fok>r(  I.I.0.0i4io<le{i]): 

l>«gj>lot(i*WMghl,»+x.y.w»ight.8/4*3); 

rolor.of(0.5.0.5.iiode(i)); 

bigj»lot(i*wfiglii.»+.\+100.y.wMglU.»/4*3); 

colot.oflO.O.O.l  .)KHlr{i]); 

l>igjilol(i*w«glil.s+.x+200.y.w»iglil.s/4*3); 


) 


plot  hoiI<hI(  x.y2.no«t«*.lowef ) 

iut  x.y2; 
iut  iKulrQ: 
iut  lower: 

{  iut  ij.k.x2.y; 
y=8priiig*x-f.30: 
x=(  1021— weight  .»*lowrt)/2; 


Si:  1 1  Sof>  M  I  OSS 


l)lotstat..s 

7(1 


plotuotk'i 

«0 


im 


plotuotlctl 

IflO 


pBft  2  of  Htnjthf.c 


plotuoded— plotuode(dispIay.c ) 


loo(ij(  lower)! 

Ioo)>i(loner)  { 

rolor.of(0.9,0. 1  .(float  )iio<le(il): 
bigj>lol(  i*weigl)l.9+x+y2, 

> -)-weight.»*2,weiglit.s/^*3)'.  ' 

))) 

j>lotnoder(x2.y2.iiCKle.lower)  plotUOlU  T 

float  x2.noOeO; 
iut  y2; 
iut  tower: 

I  iut  ij,k.x.y; 
float  maxr.niinr; 
cliar  le«»|>(.1); 
y=(iutM9|Kiiig*x2+35); 
x=(  1024-weiglit.9*lower)/2: 

(imlinax  not  le(no«le.i'ni«xr.A:uiinr. lower); 

rolurliaH  lU0.50.iiMxr.niiiir); 
looi)i( lower)  { 

sei.f  olor(  inaxr  .nunr.notJe{i] ) : 
bigj>lot(  i*weigl»t  jHfx+y2, 

y+weight.s*2.weight.#/4*3+l); 

color!  YELLOW); 

write.a«i.int(i*weight.i+x+y2— 3, 

y+weiglii.**2-M.kcla«(il); 


}} 


pl«Hiiotle<x.node.iiia«k,iija8k.up.arr»y.lower.upper) 

iut  x: 

float  iiotleQ  .innskQ  .niMk.upQ  .array 0(»iKb 
iut  lower.iipiw. 

{  iut  ij.k,y.xlj(2..v2: 
float  nisx.nun.teiiip.tenip2; 
curaolfl ); 
ya)ipring*x-f30; 

X I  =(  1024-weig)it,»*lower)/2; 

.x2=(  1024- weiglit.»*upper)/2; 
linewuiUi(2); 

fimln(Bx(array.A:niax.<;min.up|ier.lower): 
looj>i( lower)  { 

if(niaiik(i]a=t.){ 

rolor.oH  0.9,0. 1  .no<le[i] ); 

bigjTloK  i*  weiglu.s+x  1  .y  .weight.s/4*3): 


plotuode 

Mil 


1!W1 


loopi(iipper)  I 

if)  iiiaak  jipli]  =*  1.0)  j 

aet.color(max.mm.array(i](j]): 


.»2.//  iVee  .10  1993 


Put*  ^  of 


plotnode-flndniax(display.c) 

<l«wU(xl+wei6ht.»*i.y+woisht.s*3/4. 

xZ+wwgIit.s’j.y-Hipfing);}  _ 

}  • 

) 


) 

folorh«rU(«4-25(i,y  +  gO.uwx.miu) 
r«rsoii():} 


t>loHio«lek(  x.no<lp.nnia!i.ma»k.x»p.Mt«iy.lo««t  .upp«t ) 

iiit  x: 

lloNt  iKHkQ.mii»k0.in«*k.iipQ.arr»yOl»i**'l; 
iut  lower. upp«*r; 

{  iut  ij.k.y.xl_x2,.v2: 

float  ni«x.niiu.tenip.teiiip2: 

cursoH)’. 

y=«pcmg*x+30; 
xl=(  lO‘M-weiglit.s*lower)/2; 
x2=:(  l02-t-weigUl.s*upper)/2; 
liiiewullli(2); 

findmaxl  array  .fcmax.&min.xxpper. lower ); 

loopKIoawr)  ( 

ifltiiaiiik['(]=sl')( 
loopi(  upper)  { 

U(inaak.up(j]  sst  1,0)  { 

•  aet.f  olor(  niax.iiiin  .array  (i)(j) ) ; 

dr8wit(xl+weighM*i..'+'»«8ht.**3/4. 
x2+weiglit.**j.y+«|King): ) 

) 

} 


plotuodck 

!71 


i«n 


IfW 


} 

colorlmr(l(«4-256.y  +  »0.max.imn). 

ruraonO'.]  __ 

fiiKUiiax  _J 

fin<liiiax(array.niax.niin.ouU.iiia) 

fl«»at  8tray(Jlai*eJ,*niaic,*iniu; 
bit  oiita.iiis; 

(  iut  ij.k: 

iut  niaxi=0.iuaxj»0;  ^ 

iut  nimi=0,niinj=0; 

•mill  =  array[0](0); 

•max  =  array[0)[0): 

)ooi'i(in8)(  .  . 

loopj(out»){  if  (arrayO](il<  *111111)  •iiuiisarrayljllij: 

if  (8rray[j)(i)>  *iiiax)  *iiiax=iarray[i)|i); 

1 

)} 


>i:n  Nor  M  J9.M 


Poff  ^  of  Hiofiof.r 


_J 


fimlniaxiio<i<KMrr»y.mKx.imnJo<»>*r) 


flndinax-<li8pIc^.count(display.c) 

fiutUuaxuode 


flout  arT»yQ.*in8X.*niiii; 
uit  lowrr; 

{  iut  ij.k; 
bit  ninxi=0.iiiaxi~0'. 

bit  iiiiiii=0.iiiiiij=0; 

•mill  =  array  [0); 

•max  =  array(0): 

lou,-j;iow*»r)j  if  (array[j)<  •min)  •iiiiiisarrayli); 

if  (array[j]>  •max)  •max=array[j]; 


) 


di*,,iav  foui.M)  display .rouut 

{ 

liiiawi.llli(  1 );  M« 

rolor(T); 

n^tfi(  MO  .liOO.  1023 .1)20 ): 
rt>lor(500): 

rM'tUMO.OOO,  1023.021)): 
colur('t); 

wr»t<*Jnt(iilOS,003."  ".conni);} 


2-> /  I  .V»r  M  I9S.1 


Ptft  S  of  dioplof.c 


DO.TEST(test.c) 


oxtern  iut  coiinl.riKlil.goo<t.teit.*xaiii.te9t,exaniplat9.saiiiple.rtyp«; 
extent  atnict  uciiral.ii''i  net: 

#iurJncle  "de^laitloiia.lt” 

#uteln<le  "aet.h" 

#titeliMle  <iiiitth.h> 

#utchide  <st(lio.h> 
iut  nii»lakp»  =  0: 
extent  FILE  *fst; 

<  ■llF,C’K.EHROHS(y.(loft ) 
flftnt  y(sizF]; 
flont 
{  iut  ij; 
float  error(Kize]: 
iut  corrrrt.rightsfl; 
iut  rorrert^oodsO: 
loopi(o(ilpiit)  {  error(i]  =»loft(i]-  y(i]; 

if  (etTOf(i]  <  0)  errorpl  =  -error(i]; 
if  («ror(i]  <  0.5)  cofreet^oo<l++: 
if  (error{ij  <  0.2)  corre<:t.right++: 

} 

if  (forrect jood  ==  (output))  good4-+; 
elae 


CHECK.ERRORS 

in 


}n 


if  (niiatakea)  printR "Saupl*  Xad  Type  Xd\a‘*.J8nipIe,<'type); 


if  (forreet, right  »=  (output))  right+-(-: 

} 

float  eiT.oIdwsO.O.derverrrO.O: 

<IIE<K(y.iloft) 
float  y(sizi']: 
float  «io(l[ait<*]: 

{  float  niuned.hy: 
iut  ij; 

loopi(outpiit)  { 

nihned.by  =  do(l[i]-  y(i); 
w  +=  i«iiiee«IJ»y  •  niieaFil.by; 

)  ) 

fX).TfcST() 

(  iut  i: 

riglit=0;goo«l=0;; 
luot>t<Pxani.teet ){ 

MAKE.TESI'(net.inp.net.doft.i); 

FF.ED.VomVARDO; 

fll  ECK.ER  RORS(  net  .outp.net. doft ): 

) 

priiitn"Tv-at:  X3.W  X3.2t  \a".(float)right/(float)exani.te9t*100.. 

( float  )gooil  /( float  )exani.t  «M  *100.): 

fprintRf»t."Teat;  X3.31  X3.21  \x".(float)riglit/(float)exB««J«>9l*100.. 

( float  )good /( float  )exanij  eel  *  1 00 .) : 

niiMb(. 

riglil=0:goo<|s0: 


CHECK 

30 


DO.TEST 

40 


SO 


JJ.  I  J  Ser  W  jm 


t  of  tf  Ml.r 


DO.TEST-write.error(  test  .c ) 


{  iut  i; 

i^r=  0.; 

l(X)|'i(i*xaMi|>Ur»)  j 

MAKE.INP<''r(n<*t.iiii'.nHtloft.i); 

EEED.FORWAHUO; 

t'HEt'K(iK’t.outp,u«ft.iloft.4:err); 

) 

w  =  (floet)«<(rt((<lonhte)«*rr)-. 
dcrv^rr  =  (••rr  —  uliierr): 
ol«l«*tr  =  i*rr; 


) 

wril»*.»rror( ) 

I 

"  ll<■rv<•rr.3); 

writ«r.flo«t(810.70'>.180."BrTor  :  ",<‘rr,3); 

} 


porter!) 


»?n 


7n 

writP.oiTor 


.Vi»r  W  193,1 


Pttff  iofu^l.c 


(kohonen2.c) 


D.\TE:  i  Oclohfr  I  OSS 
VERStOS:  J.O 

SAME:  Tirn  Diinm^tonal  Kohontu  Map 
MODl'LE  SCMBER:  i-O 
DESCRIPTIL,,: 

ALGORITHM:  Sont  lu 

PASSED  VARIABLES:  Sant 
RETVRSS:  Sont 

GLOBAL  VARIABLES  VS  ED:  Sont 
GLOBAL  VARIABLES  CHASGED:  Sont 
FILES  READ:  Sont 
FILES  U  RITTES:  Soot 
HARDtlARE  ISPl'T:  Sont 
HARDWARE  OVTPVT:  Sont 
MODULES  CALLED:  Sont 

CALLISG  MODULES:  Sont  ,n 

AUTHOR:  Grtforp  L.  Tarr 
HISTORY: 

. . 


#<IHiue  kciion**!!  I 

/•  /  intil  1o  krtp  a.i  float  to  Hit  onfinal  atitclt  an  posmtlt  */ 

#iiirliule  <uin(h.li> 

#iuclud«  ” . .  /gr«ph .  c" 

#<|pflu«  20  /*  Htfint  »izr  of  ont  sidt  */ 

#(lcflue  inuix  sid«r*8id»  /*  nnmitr  of  ntnrant  *f 
#tl<>flue  jiimx  2  f*  no  of  inp-'ts  */ 
aO  0.3  /*  aiarlmp  fun  *f 

g  0.2  f*  inilialzalton  factor  */ 

#<iKfiue  rt  1  /*  color  nst  in  plolliuf  poinis  */ 

ii|iiUla  1000 

Kiiinu  dnisityfuiirtioiia  {8<|u«rr,  triangle,  croos); 
iut  Ik,tl.l.l2.w0.w.hl.li.vl.v.ij.c: 

float  a.al.a2.niiiiy.iii[iiiiax][jiiiax].x(jniax|.ii(iinax|.y(itnax{; 


30 


40 


/ 


a 

aO 
al 
a  I 


miiif 

m(imar][]mar] 

rfjmnr] 


cnrrrnl  alpha  mint 
alartinf  alpha  taint 


irrtfhl.i  for - to - 

r[0j  -  r  mint  r[l)  -f  mint 


ij.i.’  Sor  10  ms 


»ii 


Paft  I  of  kohontul.c 


niain(kohoueu2.c) 


a[immzj 

f[tmaz} 


V 

float  random(  ).g»uw!( ); 
iut  plotrolor( ); 

auuiu  dciisityfiinrtinns  densilyfunction: 

nmiii( )  lUivill 

/*  3tf  >9/*  »/  */ 


tiiiliali7<>< ): 

sraiid(2.1):  /*  $'i  sftH  */ 

a  s  aO:  al  =  a;  wO  =  side  /  7:  il  =■  update;  l2  =  tU:  t  =  0;  tk  s  0;  *« 

for  (i=0-  i<iniax:  i++)  | 

M[i]  =  O.U; 

for  (j=0;  jcjinax;  j++)  { 

ni(i)[j)  =  (0.5-g/2.0)  +  g*(ifloat)randoin()); 

n(i]  =  npl  +  ni(illil*ni{ilDl; 

}  I*  J  loop  *1 
ii(i]  s  n[i]  /  2.0; 

)  /•  ead  I  loop  *!  so 

«pdale_sereen(); 


/* 


V 


wliile  (a  !=  0.0) 

I 

for  (1=1;  l<=tl;  t++.lk=tk+l)  { 

/•  Ik  s  Ik  +  I:  V 

rpailiiipiil( ): 

Miiiiy  =  r[U]; 

for  (i=0;  i<iniax;  i++)  { 

>■(')  = 

for  (j=0;  j<jiiisx;  j++)  y[il  =  y(il  -  m(i)[i)  •  xfi); 
if  (y(i]  <=  miiiy)  { 
niiuy  =  y[i); 

I  "/•  M,rf  ,/  V 

)  /*  (Dll  fur  I  loop  */ 

al  s  a*(l.O  —  (fl<Mt)(/(float)tl);  a2  =  1.0  —  al; 


Ml 


inn 


Sor  10  I'm 


Pop*  i  of  kokonru'S.r 


maia-readinput(koboneu2.c) 


hi  =  f  %  si<l€; 
vl  =  f  /  sitlr: 

w  =  (iut)((eoat)wO  *  (1.0  -  (float tt/(ea«t)tl))  +  1; 

for  (h=niax(0.li!-w):  li<=niiti(si<lr— l.hl-fir):  h++)  { 

for  (v=jii*x(0.vl-w);  v<=min(m<lr-l.v|4.w)-.  w++)  { 

i  =  iijtl**v  +h;  ii(i]  =  0.0; 
for  (j=0:  j<jmax;  j++)  { 
in(i](jl  =  al*xQ]  +  a3*ni(i][i]: 
a(i|  =  n{il  +  m(i]01*nifilDl: 

}  )•  t»i  for  )  loot  *! 

n(i)  =  n(i)  /  2.0; 

)  !*  tni  for  p  loop  */ 

}  /•  tod  for  h  loop  •/ 

if  ((xpl  <  0^)  kki  x(ll<0.5)) 
folcHWHITE); 

(^laa  roiur(MAc:ENTA); 
hiiu'loi.(  h  1*  10+2.400+ V  1*10..5); 
|)(olinput(); 

if  ((t  %  t’i)  ==  1)  it|><latp.M'n>«i( ); 

)  /•  rnd  for  i  loop  •/ 

a  ss  0.2*a;  wOsO;  tls.SM;  t2s5n2; 

)  /*  tnd  wkilt  loop  •/ 

)  /*  rod  maia  profrom  •/ 


midinput( ) 

( 

mt  inaiilr; 
inaitlp  s  0; 

wltilo  (iiiaiJe  0)  ( 
inakrinp«t( ); 

switrli  (tlptiailyfunninn)  { 
caaa  a(|uarp  :  iiiaiHp  =1; 

break: 

caaa  irianglp  ;  if  (x(l]  >  2.0*faMx(0l  -  0.5))  inaiib  s  1; 

break: 

caae  rroaa  :  if  ((ralia(x[0)~0.5)  <=  0.1)  || 

(fabs<x(l)-0..5)  <=  0.1))  inakir  =  1: 
break: 

}  f*  tnd  of  roO€  V 
)  /•  rod  irkdr  loop  •/ 

)  I*  rod  rrodmptt  •/ 


32:12  Sor  30  19S8 


170 


isn 


rewliuiMit 

140 


\W 


J  of  kokoO€Oj.C 


readiuput-drawdMtribution(kohoneu2.c) 


#if  False 

l>ilU«l<>t(x2.y33i) 
bit  xS.y'i^i: 


(IrawiK  x2,y3.x2.y2-f«i ); 
lin*widtli(  I); 

) 

im 

l>iotinp<i((  (  plotiuput 

(  bit  M.hy: 

ax=(bit)< 200.0  +  x(0|  *  200.0). 

Hys(bit)(-<00.0  +  x(:]  *  200.0): 
l)igjik>i(ax.by.3);| 

I 

fllTMlt  g«un() 

I  bit  i.linras.S; 
ftont  xlsO: 
fur(  i*0;i<  ) 

xl  -fs  nimlonH); 
xl  s  xl/liiiM«;mtnru  xt:) 

l!« 


makeiupnf 

(M 


l>ig.pl<)t 


iirMw«lbtribulio«< )  (IrRWtUstrilHltioU 

xw  300  /*  i/rfimt  r  trttHk  of  plotttmf  mmior  */ 

#il«4liie  yw  300  f*  itfmt  y  rtWM  of  fMUwf  trmiotr  *f 

#fl«4lim  xhnar  0  /*  iiirrr  rsO  a  o»  irntm  */ 

#«k4tii«  ybanr  0  /*  wkfrt  fs0  a  on  »rm»  */ 

bit  xl.yl.x2.y3;  5*“ 

f*  tni  of  foot  •/ 

xl  s  xh.w: 
yl  =  yiNMP; 

x2  s  x«  xhaiip; 
y2  a  yw  4-  yhMr; 
folorl  YELLOW): 
riiiov3i(  50.320); 
rliAnitr("Kohem«B  Topology”): 

5in 

roterlWHITE); 


?J.l”  Sor  10  I91S 


Poft  4  of  tokontnS.f 


drawdistribatioti-drawline(kohcnen2.c) 

rffti(xl,yl  Ji2.y2); 

roloiif'N'AN); 

fiiiov2i('>0.rt20): 

cIiKnitt'nColioam  Hap'*); 

ro(or(BLI’E): 

Krti(0.400.200.ti00): 

cotert^'VAN'; 

finovii(2.i0.rt20): 

"HoxBaliaad" ); 
rnio'  2i(r20.t«)5). 
rharntrflapat  Dlatrlbatlea"): 

roMOLI’K): 

r.'«ti(2ni.  JOO.-JOO.tVW); 

folor(YELLO'V);  ^ 

for  (••aO:  li<ai«l^;  li++)  { 

xl  =  (xw  •  ■l■(h](Ol)  +  xl>«»e; 

yl  a  (yw  •  ni(h}[<j)  +  yl^aa^; 

x2  =  ',xw  *  (n(«ida*h|(0])  +  xbaar: 

y2  =  (yw  •  +  ylwwe; 

drawlioHfcxl  tylJiJiNfW):  dniwliiK(4:x2.4:y2ji<fc*h.i); 

for  (vai;  v<*Mr— 1:  j 

dri»wlinr(A'xl.A:yl^i«V*v+h.O):  ** 

drawliiH  A;  x  I .  A'  y  1  v +h  ) ; 

drawling  A:x?,A-y2.«'l<»*h+v.O); 

Jtawlinr(l:x2.A'y24idr*li+v,l ); 

)  /•  rnd  for  r  loop  V 

JrawliiiW  fcxl  .fcy  I  ^<tr*(*idr— I  )+h  ,0); 
drawliiMd  A:x2.Ay2^dr*b-Mide- 1 .0); 

)  /•  tmd  for  k  loop  *! 

)  /*  tnd  dioirdtslntatioH  */ 


ilrawliiH  xl*.,vl>.i2.r) 
iitt  •xl>,*vl>.i2.r; 

{ 

iut  xo.yo; 

so  =  *xh-.  *xh  a  ((xw/2.0)  *  (m(i2l[0j  +  m(i2+r)l‘*)))  + 
yo  =  ‘yb;  ‘vK  a  ((yw/2.0)  *  (in(i2j[li  +  +  yl.aw 

dr«wii(xo.yo.*xlj.*yl''); 

)  j*  fpd  drowlinf  */ 


ilrawliue 


KiO 


li:li  Sop  JO  1980 


Poft  S  of  kokomroi.r 


drawUne-plotweight(koboneu2.c) 


I  nwx  (s.b) 

iut  a.b; 

{  if  (a>b)  rfttnru  a: 

«9ia«  rtittim  b: 

} 

'  mill  (n.b) 

iut  s,b: 

{  if  (a<l>)  mtiuna  a: 
ela«  rotiuru  b; 

) 

float  raiHlom  ( ) 

I  float  x: 

xs  ((flaat)ran<i()  /  377«8.0): 

ratnru  x; 

) 


wWgIlt.S  d 


{  ult  ij.k.whKhix.iy. 
float  iiiin««i$ii(,iiiaxwWKht; 
iiiinavightsin{0)(whidi{; 
inaxwngbt3sfn(0][whi<h|; 

/*  itfi  martmum  and  minimam  lertfkl*  far  flaHwf  */ 
for(  abif  h=0;whwb<jmax;which++ ) 

{for  (k=0;k<tniax:k++){ 

if  (m{k](whi<’b]>nMx«right)  niaxwHnht=m(k](ahirh]; 
if  (iii(k)(wltkhj<iiiinarigbt)  iiiiuw(igbisiuPi](wiiicii];  )) 

/*  New.  flat  ikt  rrifkf*  •/ 

for(ohi<'li=0;wbirh<jniax;which+-f) 
for  (ksO;k<iiimx:k+-f )( 

ix=4M)-»-2?0*  whKh+(  aright ji+2  )•(  k  /si  Jr ); 
iy  =:4U0-t-(  wriglU.s+2  )*(  kOfaiitr ); 

rolorf  ploirotort  maxaright  .minaright  .mpiJtahirh])); 

l>igj>lot(ix.iy.aright.a): 

} 

/*  duplaf  a  calor  tar  ie  skew  thr  ranfr  of  iht  mtfkU  */ 
rolorl>ar(400.350.maxwrig|it  .minaright ); 

1 


,»3.  i  »  Sar  W  /9.M 


max 

?'u 


Mo 


plotwpiglit 

?WI 


9U0 


3111 


P0ff  6  of  kokoneoi.c 


piotweight-update.screen(kohonen2.c) 


TABLE  iM 

mt  |>lot(ok>r<iiiAX,iniu.\'miu«) 

IkiMt  ni*x.inin.vaJti«: 

{ 

return  8+(iut)(((valur-miii)/(in»«— Mun))*(TABLE-8');} 

rn«orliar<  x.y  .inax.iiiiii ) 

iut  x.y: 

floNt  iiiax.iiiiii: 

{  dmr  iii«x)>tring[30].Miiiiatring(30|; 
for  (i=8;i<TABLE;i++){ 
rolor(i): 

l>ig_|>lot(x+i*7.y.7);) 

sprintfl  mwntring.m .  *  .max ); 

KprintBiii.  .«triiig.**U.3f‘'.niiii); 
ciiiov2i(  x+SO.y +20 ) ; 
rh.uvtH  miniMnng); 
nnov2i(  x+^OO.y+20); 
rliaratrf  iiiaxslriug): 


writ<*J«t«*grr(x.y,*itlr.i ) 
int  x.y; 
duur  litlr{20]; 
iut  i; 

{  char  nunih«r(20]; 

sprintft  nninhcr.'Tid"  .i ): 

<'iiim2i(x.y): 

rliantr(lill>'); 
rhars(r(  number ) 

wrile.float(x.y.litle,B) 
iut  x.y; 
char  tillc[20]; 
float  fl; 

{  char  niimber(20]; 

»1>rinlf|  inimb«t."1l3 . 3f  "  .ft ); 

<'iiiov2i(x.y); 
rliarstr(  title); 
rlianitr(nuiiil>er);  } 

up<late_screen( ) 

{ilrairilmtril>iitioii( ); 
plotweiglil( ); 
rolor(<<RFiEN); 

write_float  (700.230."ilpha:”,al); 
««TiteJiitegcr(  700.200.  "Count :  ".ik); 


350 


colorhar 


33U 


writejintogor 

341 


3ft0 

w'rite.floaL 


iil><lato..sr-ri>ru 
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P»ft  7  of  tokonfK^.r 


update.scre«n(  kobou«u2  .c ) 


•w»plNifr<M«< ); 

co<ot<  BLACK): 


iVor  fO  I9SS 


Paff  S  of  kokonfoi.c 


#mcluci«''d*Tic« .  h' 
#<lcfliie  viJra  0.d5 


iiutialize(graph.c) 


{  iut  i; 

winoi>^n(“koh"); 

«louliMfuff<*r( )  ; 

SroiifigC ); 

vwwpoft(0.(ai»)(  lW7.*vHl«>).0.(uit)(  l(M7.*vid«*o)): 

«>Jor(nLAf’K): 
froiitb<iff<rr( R I  r  E ) : 

frutithnfRY(  FALSE); 

«•<••«»( ); 
liii»wi(Ub(  1 ); 

nMitp,tliiiig»( ): 

fi>r  (i=8;i<(M:i+-f )  iiiapfotor<i.i*4.i*4,i*4): 
<|d«virHMIDDLEMOU.SE); 


#if  FALSE 
<-lMrk.input( ) 

(  aliort  v«i; 
iut  x.y: 

im-wng^fZO]; 
if  (.|l«t()!=0){ 

■witch  ((|rra(l(&;viU))( 

owe  MIDDLEMOl'SE: 

tsftlr»U»lor(,VOI<SEX}: 

t=1ttmtn»tor(MOVSLY):*f 

dis.rlMi  =  (dM.cIma  +  1)%3: 

|»rintr("k«U»(l)"); 

break; 

))} 

#<^u<Uf 

#if«l«*f  p««rrcptron 
#(Miue  TABLE  Ml 

HARD.ON  RED 
#d*>flue  INDFrfERM  CREEN 
#(l<dlu«  IIARU.OEF  BLI'E 
ini)ializ>>( ) 

{  iut  ij.k: 

winop^nl  “B«t" ); 

grotifigl ); 


iuitiivlize 


clieckjuimt 


iuitirilizp 


Sor  iO  tm 


P»§f  /  of  fTOfk.C 


roior(  RLACK  h 
clfurt ); 
lilH'widUti  1 ); 


initialixe-iuune.thi.ngs(graph.c) 


foloff^'Ft.r.OWi: 

rnK>v2i(.|00.700); 

ciiani.rCItaltl-Layar  Pareaptxea'*): 

folor(rt); 

<TiK»'-2i(  10.700): 
rlisrslr<"D  ol  T"): 

riiK>v2i(  in.lUW); 

rhantH  "Oatpat" ): 

rniov2i(W.ti*i0): 

rhar»tr<''Actn«l'*): 

<'niov2i(  l-tO.OtM)); 
rhanlrCGoaas'*); 

rnmv2i(220.(Mfl); 

ch#«tr("ll*ht"): 


rniov2i(  lO.ii-lii); 
rliiinlr<"lida  3"): 

'•inov2i(  lO.OOO); 
rhantr("loda  T2"); 

rm<w2i(  10.430): 

('hanlr("lida  2"): 

rMiov2iU0.330): 
rlisratrl  "loda  Tl"): 

>'niov2i(  10.250): 
rharatrfllda  1"); 

nnov2i(  10,125): 
rhantrl  "Inputs"): 
iimptoior)  Cl  R  EEN  .-50.50.30 ); 

ft>r<i=T.\BLE/2.k=0j=8j<TABI,E+«j++) 

I  if  (j<2tM)  {i — :  inaprolor(J.U.70.i):| 
if  (j>2<>3)  ik++:  iiM|>color(j.k.70.0):) 

I 

#«udif 


iianM*  iliingsl ) 

( 


,»»;/  1  ,Vi»r  JO  I9S8 


DO 


.J 


{•0 


J 


I  Of) 


uaiuo.fluugs 


J 


P*f*  i  •/  fn/th.c 


I 


i 


naine.thing8-plotflre(graph.c) 


roloiil  );rnK>v2i(0.(5M):Hmrttr(  "tMk" ); 
cok>f(.5);cniov2i(0.(a0):ch«ritr("j*«p"): 
folor(«);«i»v2i(0.«M0);ch8r«tt< 
rok>r(T):cniov2i(0.«tK)):ch«t8tr(  "truck"); 

} 

#if  |>«Tn>‘ron 
#<IHliic  wright.a  (i 

x.>'.*rray,iup.out,iu»*WMslit.iiun««»l'‘  > 

bit  x.y; 
flout  array  Q; 
bit  inp.oiit: 

float  nMX«Mgbt.inbi«Mght: 

(  bit  ij,k.wliicii.ix.iy; 

/  *  (id  marimain  »ni  iniaimani  reifUs  for  floUtnf  */ 

fiiK)i«ax(array.A;niaxwghi.i:minw«ghl.o«i,iiu»)'. 
rolorbar)  x.y  .maxweiglit  jniiiwviglil ) ; 

20;  y  +=20; 

I*  Norn,  plot  ti*  mykla  hnt  */ 

for  (i=0;i<iDp;i++){ 

for  { 

irt.cob)r<inaxwctght,raini*wglit.anray(i*<wtpwt+j)): 
hi(Uilol(j*wglit.»+x,i*w«glit.8+y,w«ghl^- 1 ): 

}  ) 


I  in 


plotwpiglit 


Ijn 


tM 


ptoUKxMx.y.nodr.lrng)  plotUOcle 

mt  x.y: 
float  iKxIrQ: 
bit  l«Mig; 

{  bit  ij.k: 
loopi(lfng)  { 

!*-l.rolor<  I  ); 

Kig.plot(i*12+x.y.l0); 

} 

plotfitHx.y.lii.lo.iK)<lr.l<*ng) 
bit  x.y; 

float  lii.lo.iKxlt'Q; 
bit  l•‘ng; 

I  bit  ij.k; 
loopidmg)  { 

••olor.on  lii.lo.nod«’(i)); 

•>ig.i»lot(i*  12+x.y.lO); 

1 


JidlSor  W  I9SS 


Poff  I  of  troph.r 


} 

color  .n((  hi  .lo,valiH>) 
float  hi.lo.valiu*-. 

{  roMINUriERM). 
iflvaitic  >  hi)  folor(ll.\RD,ON): 
iflvaioc  <  lo)  color) HAHD.OFF); 

) 

plotiio<lri(x.>'.nodv.)cng) 
iut  x.y: 
int  noih^): 
mi  Wng; 

{  iut  ij.k: 
loopidcng)  { 

9ci.co(Qr(  1  .U,0.0.(fliMit  |iio<lr{i]); 
hij4>lot(i*  l2+x,y,10): 

} 

) 


plotar*-colorbar(graph.c ) 

color.of 


pluluotU’i 


im 


Rndinaxiwray.maxjiua.ouU.iiM)  fiutliUHX 

float  iumiyQ.*inax.*min: 
iut  niiU.ina: 

{  iut  ij.k: 

*111111  =  •fr*y(0); 

•max  =  arraylO):  <*® 

loopi(o««ta*ina)(tf  )an'8y(i)<  •min)  •niinaarray(il; 

if  (arTay(il>  •max)  •maxssarrayp); 

} 


) 

plotjloJr(  ); 


cniorhar)  X.y  .max.iiiiii )  <'oloi'J  >111 

iut  x.y: 
float  max.min: 
i  iut  i: 

r.ltar  niaxa(rinf(20).minatrinK(20]; 
for  (i=«;i<  lAULE+*-.i++.i++)( 


JJ.  I  I  .Vor  to  toss  Pafc  ^  of  inpt  r 


I 


roJorti): 

apriii(((niM*tring.‘*X3 . 3f "  .mm ): 
uprintn  iiiiiMl  ring."%3 . 31"  jfiuu). 
/•  eoioiiBLVE); 

f-f ->0. 90.  J  0):  •/ 
fotec(  YELLOW); 
niK>v2i(x..v): 

(hMKtriniiiiotriiig); 

/•  c<>Uf(BLrE}: 

k»tj’oti(r+l9.9.i^i0.a0.  tO): 
cMor(YELLOtt):*l 
riiiuv2i(x-f  IM.y): 
rfaa:iitr<niMatring): 


»i.roioii  nim.iiiiii,va|iH*) 

flimt  iimx.niiii.valuv; 

{  Bimt  ixfrrnl: 
bat  raU: 

if(value  >  ntml  vait«v  a  mm; 
if(v«lnr  <  mini  vnliw  a  min; 


prrfvnt  s  (vnluv'-nun)/(mm— min)  *  (S'Mit)(TABLL-'2); 

foU  a  (bat)prtrenl  +  3; 

ru(or(ro(x): 

) 

higjtlot(x..vjiisr.of.(>ol) 
bit  x.y.mar.of.i>ot: 

{  lin^i<)th(MS«.or.<)ot); 
ilniai((  X  .y.x  or+*i*f.of.«lo» ) : 
liii'-widtlii  I ); 

) 

higj>lo«2(x.y.ww.hh ) 
bit  x.y.ww.lili; 

I  liin‘wi<lth(nw); 
ilrawii(x.y.xjr-fMi): 
liiH^iilllt(  I ); 

} 


lirmriti  x»(iirl.y*(ari.XPnM  .yra<i) 
bit  .vit«rl.yxtart.x<fn<l,y<>ti<l: 


colorbar~<lrawit(graph.c) 
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3» 


set. color 

nn 


>40 

liig4»Iut 


l)ig.l>lt>t2 

7»*» 


<  Irani  t 

J<(0 


ja-tiNor  to  J999 


Ptff  i  of  fTHfh.C 


drawit-drawit2(graph.c) 


(Ir»w2i( 


(irmrii  2(  xatmti  .y»«iirt  .yntd ) 

float  ut«rt.>-8tart.x«nd.ym«l; 

{ 

iiir>w2(  xn»«ft  .y^tart ); 
Jr»w2(x«id.y«nl): 

) 


(lraTvit2 

jrn 
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main(  Error.surc.c ) 


DATE:  -•  Scpi  198S 
VERSIOS:  t  o 

SAME:  Error  Sorfart  DtmoH.iInlinn 
MODULE  SrMBER:5.0 

DESCRIPTIOS:  Error  Marfnrr  griKnilion  loop 
ALGORITHM:  FtrilforTonl  BackpropagaUon 

PASSED  VARIABLES:  Soar  lo 

RETURSS:  Sont 

GLOBAL  VARIABLES  USED:  Sont 
GLOBAL  VARIABLES  CHASGED:  Sont 
FILES  READ:  Soar 
FILES  WRITTES:  Soar 
HA  R  DIVA  RE  IS  PUT:  Soar 
hardware  OUTPUT:  Sont 
MODULES  CALLED:  Sont 
CALLISG  MODULES:  Sont 

30 

AUTHOR:  Grtgorp  L.  Tarr 
HISTORY: 

. . . . 


floMt  wtsl..w3sl.,;c.>'.outpiit.do(t.«rroraa  1., targets  0.05: 
floMt  oJ<l<»rror; 
fl<Mt  r«n<loni  (); 
iitint  ramloiiixl ): 

vi<|oo  0.45  .-M) 

#iuriu(l<>  "•turfaca-c” 

#iucluU«  <liiiiv.li> 

OR  AY  1000 

#aoftu«  c:kay2  looi 

flcmt  nctos  0.3: 
iut  wruiigs-l: 
iut  whirlisO.wliirh'isO: 
iut  rli^kjiKMMpO: 
iut  roiiiii; 

fliMt  *lArt.wl3lKrl.w2;  4n 

iiinin( )  lUaiu 

{  iut  i: 

flont  ol<{wl.olUw2: 
kMig  now; 

oraml(linHA-now)  %  37  ): 

«irf»co(TRl'E): 

iiiit.nioiiwK ); 

initpcolorl  lOOO.IOO.lOO.lOO): 

text  |>orl  1020.0,220):  .ii> 

|>ag*^olor(  1000): 

l<‘Xlrotor<(.'YAN): 

«yoli>iii  ("el»«r"): 


>J:U  Sor  10  1989 


Pagt  I  of  Ermr^tarr.c 


lOTniii— feadforWardf  Error.surc.c  ) 


nietiu( ); 

inov»(irt.w2.-<.); 

folor(YELLO\V); 

initislnr  (): 

for(i=0.coiint=0:i<200;i++){ 

wliiie  (errors  >  target  )  { 
niakeinpnt( ); 
feedfurwaf<l( ): 
move(wl.w2.4.n); 
olJwl  =  w(;oWw2  *  w2: 

if  <which2  ==  0) 

harkpro|>agate< ): 
else  l>aekpro(>agate2( ): 

if  (wliirh2  *=  J)  eolor(YELLOW): 
else  cotor(KF.I)); 

<lr8»r(wl.«e2.4,); 

if  (Trhieli2  ==  1)  foUir(OAN); 
else  roloriYELLOW): 

pnt(ol<lw  1  .oltJ  w2.i  .0) : 
dlteount  %  5)  a=  0){t4^(i): 

8ereen.two( ); 
screm.o**e( ); 
inove(wl,w2.'l.):} 

ill eheek.mo««se(  1  LEFTMOVSEf  hnmk: 
co«oi++:} 

printa"start  X3.St  %3.M  Va  sad  X3.H  X3.7t  ", 

start  .w  1  aUrt.w2.w  1  .wS ); 
iuitiaiise  (); 

errors  =  l.O; 

)} 

iiiilialize< ) 

{wl  =  raii«lonr( ); 

«2  =  raiiiloni(): 

iiiapeolori  2 .0 . 1 28 ,0 ) ; 
start_wl=w|; 

8tart.w2=w2: 

) 

/•  Errd/orwarrf  htckproptfiion  rult$  */ 

feerlforwarili } 

I  float  temp: 

=  wl*x  +  W2  *>•  +  1.;  r  <kria  i»  osr  */ 
ootpiit  s!  stgiiK>id(t»Hiip): 

) 

float  Heltax  =  0.0  .Jrlfay  =  0.0  ; 
float  oirlileir  —  0.0  .oUldely  =  0.0: 
float  moiiH'Ms  0.0: 


TO 


laj 


W 


fr(»<lfon''ar(l 


UK) 


».».7V  y»r  10  lO.iO 


Ptft  i  of  EiT»r..rBrr.f 


feedforward-test(Error.surc.c) 


bArkpmpAKitM ) 

{  flmit  (h  i; 

olddrtx  s  nionM^  *  (Mtiix: 
oldddy  s  ntonini  *  (Htay: 

del  s  output  *  (1.0  —  output)  *  (do(t  —  output); 

<i<*ltax  =  nri*  *  del  *  x; 
dWtay  =  net*  •  <W  *  y; 

«l  s  wl  +  (kdtax  -f  olddrix: 
wV  a  wj  +  (Wuy  +  oiddriy: 

) 

flout  V.Q.R.cYrorpriiiip; 

flout  .41  a  0.1.  42  aU.O  JV3  sO.O  .44  a  0.1.  AS  s  O.OS. 
flout  r.fpriflH>.(’rror.(M»t=0.,d«H«r2a0.; 

liark|iropugatr2( ) 

{ 

I'  a  ontpiif*(  1.0— output): 

V  a  X  «  (43  *  w|  +  4.5  *  dHwl)  4-  y  •  (43  *  w2  +  AS 
rprilia  a  It  •  V; 

(TfOf  a  2.0  •  (doA  -  output): 

Qalt  •  mot; 

(‘tTorpriine  a  -2.0  •  fpniiiK 

Ra(t  •  (morprin«F4«4Tor  *  (I.O— 2.0*oatpu()  *  V); 

<Mwl  a  (Mat  -  42  *  wl 

-  44  *  drlwl 

4-  (41  •  q  4-  R  )  •  »: 
ddw2  a  dclw2  -  42  *  w2 

-  44  •  (Ww2 

4-  (41  •  Q  4-  R  )  •  y; 
wl  a  wl  4-  <Hwl; 
w2  a  w2  4-  (Hw2: 

pHntR*'wl  X3.3f  *2  XS.2f  tfalut  U.M  ««lv3  13.2  Xu", 
w  1  ,w2.dHw  1  ,d«4w2); 


J 


t«u(i) 
iut  i; 

I  flout  (M; 

WWMIg  a  0: 

X  a  pix: 

>■  a  ply; 

doA  a  «iKniotd(2.0); 

(mlforwani( ); 

«M  a  doA  —  outpttt;«Tfon  alupiarridel): 
»y«t«iii("elaux*'); 
irxttiort(  TOO.  1020.0.220): 


iJ:l4  Xor  JO  I9SS 


ba<'ki)ropagato 

iin 

120 

)>acki>roi>agAfc2 

d<4w2); 

ISO 

MO 

test 

mn 

p0ff  i  firor.flWfr.f 


te*t-prop«gate(Error.surc.c) 


(irinlf  Xd  Trial  Xd  \a'',couni,i): 

priiitf  (**  doft  Mtpat  arror  Va**): 

prmtr  ("1  X3.3t  X3.3f  XS.3<  \B**.ilort .output. 


X  s  p2x: 
y  =  p2y: 

doft  =  iiiginoid(2.0); 
fmIfonrardC ); 
cH  s  doft  —  output: 
priiitf  ("2  X3.3t  X3.3f 
mvtn  +s*  Minan'fdri): 

X  s  p.ix; 

y  *  p3>; 

doft  s  •igiiiQid(2.0); 
rmlforwartll ): 

<M  s  doft  —  output: 
i^Tom  +SS 

priiitf  ('■3  X3.3t  X3.3t 
X  =  p-tx: 

y  =  Hy; 

doft  a  MgmokM-X.O): 
faadforwwdf ); 

<M  a  doft  —  output: 
•■rrora  -fa  aiiiurr^drl); 
priutf  X3.3f  X3.3f 


X3.3f  Ka" .doft .out put.  drS); 


X3.3f  \a" .doft .output.  d«i): 


X3.3f  Vb". doft, output.  iM): 


X  a  p-lx: 
y  a  pSiy: 

(kift  aaignioidf— 2.0); 

frnlforwunM ); 

<H  a  doft  -  output; 
mon  +a  M|uiu*(.M); 

priutf  ("S  X3.3t  X3.3<  X3.3f  Vs" .doft. output,  dai); 


X  a  |iAx; 

V  =  pOy: 

doft  a  aignmidf— 2.0); 
fnulforwardf ); 

(M  a  doft  —  output: 
rrron  -fa  squara<d<'l); 

priutf  ("«  X3.3(  X3.3f  X3.3t  \B".<loft.outpiii.  iM): 

priutf  ("MS  arror  X3.3f  \*".«‘ror8); 

) 


propasaH) 

(  fluHt  iM: 

(k-l  a  oiitpul  —  iloft; 
wl  —a  iKla  *  del  •  x; 
w2  —a  iieta  *  d«d  *  v; 

) 


iro 


IMI 


300 


propagate 


Iln 


*.!:I4  .Var  W  1999 


Pttft  4  of  Error jtwrr.r 


propagate-makeinput(Error.8urc.c) 

UiakciupiU 


(  hit  which: 
which  =  n«nii()  % 

•witdi(  which) 

{  CMw  0:  X  a  plx: 

y  =  ply; 

'h>ft  =  MSliioi<)(3.0); 

Iwvak; 

««  1:  X  =  P2x: 
y  *  p3y: 

•loft  s  iiiginoiHlS.O); 

brwtk: 

CMm  2:  X  =  p3x; 

y  *  pJy: 

‘h’ft  =  •i8nini<<(2.0)- 

hrcMk: 

3:  *  *  Nx; 
y  =  Hy: 

'h>A  a  ai(iiio«<l(-3.0): 

hnwik: 

«*•*  4:  X  a  pJix; 

y  *  Piy  : 

»  Mpnoidl-S.O)- 
break; 

S:  X  a  p«ix; 
y  *  P«y; 

<h)ft  a  aitnioiWl.S.O)- 

break:} 


Wo 


»n 


lei 


I 

Boat  nuiOotii  () 

}  float  x: 
hit  y; 

y=(r«ml()  9r  4000); 
xa((float)y/4000,0~0.5): 

X  a  X  *  4.0; 
xwtarn  x; 

) 

float  ritii«tontx| ) 

{  float  x; 
hit  y; 

yalmndO  %  200); 
xa((«oat)y/2no.0  -  0,5); 

ix^nra  2.‘x; 

.’2.14  ,Vor  W  I9SS 

.4  of  f rror  *«rr.r 


ataJceinput-check.moiue(  Error.surc.c ) 


} 

iuil.ino<wp< ) 

(  nwprolott  1000.100.100.100): 
iiiaprolorl  1001.100.100.200): 

M  iniH^EMOt 'SC): 
.plavirHlCrrMOI'SC): 
•l«i«>vioKHlC.'UTKIOl'SE):) 


rhrrk_nioti«a< ) 

{  almrt  %'«1: 
iut 

flont  t4Mnp.xtraip.yl^np: 

Sfretiifooftl  x«.y«: 
iut  sfWt: 
fl<mt  xvaliM*: 
eiuur  niniMg<^20]: 

ir  l<|lMt(  )!s0)(  cunoffi  );inniu< ): 

xs=gMvii|nstor<  MOt'SCX); 
ys=grtv«iuaior(  MOUSEY):  . 

xxalnos  (flQ«tM»-ll0)*0  02; 
if  (xtniw  >  2.0)  xvslur  s  2.0: 
if  (xvaliia  <  0.0)  x\-iiliH>  =  0.0: 
arhrt  a  (int)((flu«t)yH/20): 

■witch  (qrc«d(Ji:v«l)){ 

ciwe  MIDDLCMOl'SC:  if)%Klaal){ 

xs=  get  valuator!  MOt'SCX): 
yaaget  valuator!  MOl'.SCY ): 
w2  a  |aaNt)!xa)/l048.*  2.0; 
wl  a  !ao«t)!y*)/7tJ8.»  2.0: 

} 

brrmk: 

CUM  LEFTMOUSE:  /•,/(r*lssl){  )•/ 
breuk; 

cua*  RIOIITMOUSC:  il!valasl){ 

il!irliicb2  as  0)  { 

awitch!aelort){ 
cjw*  0;  nela  s  xvaliie: 

Write.float  (10.120.110."  gta;".iieta.CTAN):breiik: 
cm«  -S:  nioinen  a  xvaliie: 


:i:l4  Sor  iO  I9SS 


iuit. mouse 

170 


(•lieck.moiise 


imt 


M 


300 


310 


p0ff  6  of  Error^Morc.c 


check.mouse-write.float(  Error.surc.c ) 


cbc  { 

cmtm  0: 
cmm  R: 
CM*  T: 

CMM  <(; 

CAM  -5: 

)) 


writejloat  (10.100.110.“ 

)} 


No*  :".iiwn)tfn.C'VAN);break; 


•witrli(srl4^t){ 

A1  =  .xv»tur; 

(10.180. 110.' 
A3  =  xvaliir: 
wriu.fkwt  (10.160,110.' 
A3  =  .xvalu«-. 
wrUr.Oo*!  (10.140.110.' 
A4  =  xvnlit<>: 
writc.floal  (10,130.110.' 
A5  s  xvaiu^-, 
wriU.floM  (10,100.110.' 


*1  :".Al.CYAN);brt;i»k 
i3  :“.A3.CYAN);br«ik 
AS  :“,A3.fYAN);br*«»k 
A4  :".A1.0  AN):brBiik 
iB  ;“.A5.CYAN):hr«itk 


•witcblarlfTt)! 

CMC  4:  which'i  =  1: 

if  (wliichS  0)  <rnt#.strmg(  10.80,1 10.“first  0x4ar“); 
el«e  wnt<>.!itrmg(  10.60.1 10.“S*ea«d  0r4«r“); 

iiiMMil  );hreak: 

cMe  3:  iflxvalue  =ss  U.0)lh«.rrtuni  st  LEFTMOl’SE: 

tibw  {color( BLACK );cI«m():  urfaM(FALSE):mraa():) 


brenk; 

CMMi  3: 

iflxvaiu^  ss  0.0)(wkiln(viUs«() 

wluMvailsI)  qrrad(l:\-iU);) 

oIm  *xit(0); 

br«Mk; 


) 


}  /•  */  if  V 

]  /•  fni  of  7 

}  /•  fnd  of  ,f  7 


riinoni ): 

rx  turn  llw  relitrii; 

) 


«'rtl<‘,nrMii(x.y.l.tille.Ii.a.xo)or) 
ilik  x.y.l; 
rbiur  litl<^30]: 

fliMit  fl: 
iut  a.rolor; 

{  rJuur  nHmh«>r[30]: 

I  s  (iut)((fla«t)l-fl00.  ); 
afiwnjUr**^)'. 
*|>riiitniiUiiilM*r."X3. 
color!  a.tolor); 


22:H  Xoo  JO  ms 


3?0 


XW 


MO 


aan 


wrifo.0oat 

3^1 


.irn 

r«ff  7  o/  fm>r.ji«rp.f 


WTite.float-«scale(Error.surc.c) 

5.y-5jc+l.y+15); 
roMVKLLOVV). 
siKalo<x  (-50.y); 

)  1 ); 

fniov2i(x— 2.y): 
rliantr(till<>); 


rliaralr(iiiiiiibrr);  } 

> 

«rri(4*.9tnng(x.y.l.titlr) 

writc.striug 

iitt  x.y.l: 
clMr  lill«(20]; 

{  rJuur  nunilwr[20]; 

1  s  (iat)((iio«t)l/vi<l^); 
rok>r(tJ|; 

r»^tfi(x-5.y-5jc+l,y+15): 

folof<4): 

r«^ti(  X— "i.y — .'J.X4.I.J  + 15): 
<'nvjv2i<x.y— 2): 

3f*0 

» 

rii  ant  r<  title); 

) 

» 

nwwiO 

{  riintoffl): 
it  («liidi2  as  0){ 

wfiujlaiu  (10. 120.no.-  ft«;-.iiet*.nR.4V): 
writ«.floiii  (10.100.110.-  ll<Ni;-.nimiiFn.C.HAY); 
wrile.»triiig(  10.80.110,-  Ftrat  Oraar"):) 

luemt 

«be  { 

srile.atriiigl  10.80. no.-  Sacoad  Ordar-); 
writr.float  (10.180.110."  il  :-.AI.URAY2); 
write.doat  (lO.HW.llO.-  82  :--A2.t:RAY2): 
write.float  (10.140.110."  83  :--A3.C;RAY2): 

400 

riinon( ): 

write.floai  (10.120,110."  84  ;-.A4.nHAY2); 
writa.Boat  ( lO.lOtl.llO."  88  :-.A5.t;RAV2):) 
write.»tring(  I0.ii0.no,"  fart  Claar"): 

write.atringl  10.40,1 10,"  lalt  Quit  "): 

» 

i 

4IU 

aaralefx.y) 
iut  x.y; 

.wule 

{  iut  i: 

W 

X  3  X  +  50: 
linewi(llh(2): 
rolof<\  KLLOW); 
loo|>i(  10)( 

niovi»2i(x+i*l0,y+5):dr»w2i(x+i*10.y):*lraw2i(x+10+i*10.y);)  <?ii 

(lrair2i(x+i*10.y+5);  ) 


9 


A'or  JO  JOSS 


Puft  S  of  Emr^ftrc.e 


(surface.c) 


I 


1 


I 


. . . 

» 

•  DATE:  i  Ocioitr  19SS 

•  VERSIOS:  i.O 

• 

•  iV.l.Uf;  Surftct.c 

•  MOUC  LE  vetfSEft: 

•  DESCRIPTIOS’:  Sarf»ce  Grnfrmior  for  Emr^strc 

•  ALCORITHM.  Somi 

•  PASSED  VARIABLES:  Sont 

•  RETl  RtyS:  Sont 

•  GLOBAL  VARIABLES  USED:  Somf 

•  GLOBAL  VARIABLES  CHASGED:  Sot,, 

•  FILES  READ:  Son* 

•  FILES  IVRITTES:  Son* 

•  HA  R DU  A  RE  ISPVT:  Son, 

•  HARDWARE  Ol'T^VT:  Son* 

•  MOnVLES  GALLED:  Son, 

•  CALLISG  MODl’LES:  Son* 

« 

•  Al'THOR:  Grtforf  L.  Torr 

•  BISTORV: 

• 

. . . . 

/•  Bepmntnf  of  Strong  Jilt  eompontni  trror.c  */ 

#iuch««lo<gl.li> 

#iaehi<ln<  df-v  ic«.  h  > 

#iur  lii(le<iii«lli .  Ii  > 
sitr  50 

#«l<-flue  li<'||o<a|  |>rinll1( "hallo  Xd\ii".«): 

#<l«4iu«  (5.0) 

inrrr  (-2.0*  sisrt/sii«) 

flout  gptx(); 
flont  rngnioidO; 
floNt  tlH*.funclion( ): 
flont  s«|ii<irH): 
flont  ninx.niin: 
flont  nrrayfaitfKmxr]; 

flont  plx.|>l>’.p2x.p2y.)>3x.p3>,p-tx.pdy.p5x,p5y,p0x,p0y; 


III 


4M 


flool  httfhifr.f) 

flont  r.f: 

I  flont  :: 

tot  t.j:  wi 

I  s  (int){  r  /  I  S  -  20  ): 
j  *  (  oll(  y  /  t.S  -  20  ): 

■  =  •rrnt{i}[j}; 


« 


22:IS  Sor  iO  I9SS 


Pnf*  I  of  nntfnc.c 


•urface(surface.c) 


rrtani  :: 

) 

7 

TABLE  512 

#dt«ilue  lip  (0.50  *  iiicr^) 
iut  grtcolor( ): 

.. 

,  mirfnrrifMt) 

iut  Tmt: 

{  SoRt  x.xu.yn.y: 
iut  ij,k: 

<  'oord  purrnyfllpl: 

*  C'olorindfX  iarray{-4]; 

ctinofTl ); 

,  iflfMt  ==  TRl'EX 

fillj>oinU( ): 

for  (i=0:i<.<izr;i++,.x=:g**tx(i))j 

for  (j=0u<'*wu++  y=*‘’‘’'0)){ 

array(i]y]=  (tlir,fuijctioii(x.y)): 

)} 

IHii«li«iAx(arrsy); 

giniK): 

gco<ifig( ); 

lw(is'rAHLE/2.k=0j=«j<TABLEa++) 

J  if  U<2!>d)  {i — ;  ni»|Kolor(j.0.I28-i/2.i);| 

^  if  (j>25r>)  {k++;  iiiBp<'okjt(j,k.l28-k/2.0);}) 

folorfO): 

‘  rlrwf ); 

rolr>T(nLAf'K); 

I  rUnrO'.) 

grrrrnjjnH  )'■ 
zbiiBotiTRl'E); 
zrlrnrC ); 
drnwflooH ): 

for  (i=I:i<«ir-l:i++){x=g*»tx(i):xnssg«*lx(i+l ); 

for  (j=  I u<*i*r-  l  j++  M J -8^‘*<j):yn=«rtxO+I ): 

piirr»y(0l(0j  =  x: 
pnm>(0|(l]  =  y; 
p«rrsy(oj(2]  ® 

iwTTsy  (0)=grtf  ok>r(  Rmiy  (iJli) ) : 

pRrray(l)(0}  s  xn— up: 
p«rr«y[l|(li  =  y: 
p*rray(lji2j  =  »rray(i+l][i); 
mrTay[l)=grtcok>r(MT«y(i+ 


iVor  HI  tm 


Paft  i  of  oorfmrr.c 


surface~nonnal(turface.c) 


p»iT«y(2]lO]  =  xn-up; 
l>arrny[2j(lj  =  yu-up; 
p«rray(2](2l  =  arrayfi+lJlj+l); 
iwray  [2]=g«tcolor(«K*y(i+l)[j+ 1]): 


)} 

ib«ffrr<  FALSE). 
rnnioii( ); 

J 

fillj>oinls( ) 

I 

l>l  xsO.fi: 

|.ly=2.5: 

p2*®!*lx  +  raii<lomx(); 
pJyaply  +  rMMkMuxO: 

p3xsplx  +  raiiHo«nx(); 
l>.1ysply  +  rau<loiiu(); 


parray[3](n]  =  x: 

|>array[:tj(l]  =  yii-up; 
parray(3ji2l  -  array(ij{j+|]: 
iarray  (3]=g<ftoo)of(  array  (i](j+  i] ); 
«tiir(  'l.iiarray.iarray ); 

color!  CYAN): 


plx=3.n-. 

|►^y■=1.0: 

l»5xs|»4x  +  raiKloiiixO; 
pAysiMy  i-  raiitloiiixl ): 

|ir)xs|»4x  +  randonixl); 
ptfysply  +  raudonixO: 


) 

grtcolor<x) 

fluMt  x; 

I 

C'oKjrindcx  light; 

light  =  (<oloriii<lex)((TABLE-20)*(*~nnn)/(max-niin)); 

rr^urn  (light+«); 

} 

norma  l(x.y.x) 

float  *x. 


m 


fiU.poiuts 


lao 


141) 


gctcolor 

ISII 


normal 


iVor  .fO  193S 


Pxft  .#  o/tar/arr.r 


uorinal-<lrawfloor(surface.c) 


{  ftoMt  iiMg.s.b.c; 
b  =  ‘v: 

c**i; 

mag  =  s**+b*l>+r*r; 

mag  s  (8(Mit)s<|rt((donble)niag); 

•x  s  a  /  mag; 

*y  =  b  /  mag; 

*1  as  f  /  mag; 

1 

ITO 

flout  gatx(i) 
htt  i; 

(  flout  x; 

X  as  iiifn*  *(flcat)i; 

X  as  X  +  ttAtl: 

retitru  x; 

I  IM 

flout  «ign<oid(y) 

floot  y  ; 

i 

vctttful  l.O/(l.O  +  (floot)ex|*(-(<k>obfci)(y)))); 

I 

Jort  atpiarv  (x) 
fliiMt  x; 

(  fioflt  y; 

y  =  X  '  x; 

rutnrn  y  ;)  '<« 

#dt4iue  a  -2. 


float  i(H>,fun(tioii(x.y) 
float  x.y; 

{  float  irrr,ml.m2.w3.err4.m5.aTTa; 

wl  as  nqiiatHMgnioidU')  -  aigiwoi<l(  I .+  plx  *  x  +  ply  •  y)): 
arT2  =  K|uar«<M8nH>id(l.)  -  »igii¥)i<l(l.+  p2x  *  x  +  p2y  *  y)|; 

s  stiuarHaignioidi  !■)  -  sigmoid)  1.+  p3x  *  x  +  p3y  *  >•)); 
wt  s  sqiiaHsignioidl  — 1.)  -  Mginob((l.+  pdx  *  x  +  i»4y  *  y)); 
mb  »  sqnarHsiSUioidl-l.)  -  sigmoid)  l.-f  pbx  *  x  +  pby  *  y)); 

ss  squaHsismoid)-!.)  -  sigmoid) l.-f  pdx  *  x  +  pdy  *  y)); 
m  ae  arrl  -f  w2  +  »Tr3  +  w4  +  enb  +  ettd; 

TOturn  (2  *  w): 

) 

dravflnOfi ) 

1  rolof  )BU'E); 


KMI 


ilra\vfltH>r 

lin 


ii.lf  .Vor  10  103$ 


P*ft  4  of*urf»rt.c 


drawfioor-8creea.two(8urface.c) 


cofc»c<C\‘AN): 

niovi^ — — 5.  .0. ) : 

«inMr(— 5..5..0.): 
dnt«(5..5..U.); 

Hr»w(5..— S-.O.): 
dr»i»(— 5..— 

niovr(-iS  .0..0.); 
dr«w((i.,0..0.|; 

nwvr<0..— rt..O.); 
dmw(0..tf..0.): 

nwv»<0.,0..— 2.): 
dra»(0..0.,5.); 

m)ov(rt.OI.O..O.):rh<inilr(  "X"); 
riiiuv(0..tt.U  1 ,0.  ):rh«nitr(  "T" ); 
rniov(  0.  .0.  .li.O  I  );chnfiilr<  "X" ) : 

) 

Rud  II  iax(  array) 

#(kiAao  loopi(ON)  fnr<i=0;i<ON;i-f+) 
loo|)j(ON)  for(j=Og<ON‘4-f+) 
jjkilfjfiiM  loopk(ON)  for<ks<];k<ON;k4'd’) 

float  arrayQ(Mx«|; 

{  ink  ij.k: 
iat  iiiaxisO.niaiOsO: 
iut  niiiiisO.iiiinjsO: 
min  =  array(0|[0j; 
max  =  array(U][0]; 
loopi(aix«){ 

loopj(aii»)(  if  (array[j](i]<  min)  niinsarray[i]fi]; 

if  (array[i][i]>  max)  niax=array[i](i]; 

) 

}  printf  ("U  Xf  \B".max.niin): 

) 

wrwi  oiiH ) 

ortlio<0.0.(floNk)XRIAXSC'RF;EN.0.0.(float)YMAXSC'REEN.-1.0.l.0); 
sruli'plb( 0x000 .Ox  FOO ) : 

vi<>w|N>rl(0.(int)(  10*<J3.*vul»«),0.(uit)(7H7.*vid«)): 
fifrapM-tivi^  I50.1.0.2.0.-2.0); 
lookat(2..0..!i0..U..0..0..0.0); 


1 

nrrarn.twoi ) 

{  iut  i: 

float  Ml.rixM.lop.IxMloni; 

— l.:ri$li(  =  5.:l)Oitoiii  =  — l.:lop  s  R.; 


310 


730 

iiucUiiax 


740 


7»0 

scTcou.oue 


TOO 

StTOfU.tWO 


«.7.5  Sor  JO  toss 


Poff  5  of  ttrfort.c 


screeu.two-screen.three(surface.c ) 


f 

I 

i 

V 

\ 

\ 

I 

1 


ortlK>2(  k'rt, right, bottom,  top ); 

niraoiTi ); 

cok)r(100*>): 

vir«’port((iut)(Tnfl.*vi(.l(>o|.Uutl(!)0(  .''viiltY)) 

.(uit)(5OO.*vKl«>),(iu»)(T00.*vU!«j)): 

floBri ): 

foh.rOEM-0\V): 

niovp2(  lofl.i  ):(lraw2(  right.]  j: 

:»r<i=  -.l;i<4:i++) 
iiiove2(t.l>ottoni);dra<r2(i.to|i): 
cok>r<BLI'E): 

flimvaip  lx.p  ’.y  );rhaf*tr<"*";; 
cniov2(  p2x.p2y  ):rh8ri»tr< 
f  mov'.i  p.1x,p3y  l;ch«fatr(  ); 
rolori  BLAt ‘K ): 
riiiov2(  Hx,p4y  );rhantr< 
rniov2(P''>x.p-')y  );rh«r;tr( ''  •" ); 
riiiov2(  |>tfx.p<iy  );rhantr( 
rolo«(7); 

Siioi*e2(0.0.-0.5/w2): 

dr«w(.'i00U..(-("l/w2*«)00.)  -  0.5/w2)): 
inow2;0.0.-0..Vw2); 

ar»w(-!i000..((wl/w2*«)00.)  -  0.5/t.2)); 

«iiaon(); 

) 


•f  f«^.lhr<^ ) 

{ 

ortlio2(0.,I023.*vi<l«i,0.7i>7.*vi<l*w); 
virwport(0,(iut)(  1023.*vi<lvu).0.(iut)(767.*vi(i«o)); 
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ABSTRACT 


An  viiviruiinionl  in  <lovelo|)^  for  tho  xtinly  of  dyiiniiiic  cliHiigtv  in  |>AlU-riiii  of 
weight  And  nwlv  vaIiio?i  for  artificial  neurai  nelwitrks.  Graphic  rrprraciitationa  of 
iicnral  network  internal  atatoa  are  displayeil  naing  a  high  reaohitioii  video  terminal. 
Palterna  of  node  (iriiigs  and  ch'.iigea  in  weight  vectors  arc  dis|ila3'e<|  to  provide  in¬ 
sight  during  triiining.  Pour  pattern  recognition  prohleiiis  are  applied  to  four  Ivin's 
of  artificial  neural  ni'l works.  Using  simulated  data,  a  simple  ilisjoint  region  rla.ssi- 
fication  problem  is  develo|)ed  and  examined  using  a  Kohonen  net  and  a  innllila^'cr 
fee.lforward  liack  propagation  (MFD)  network. 

A  MFB  neural  network  is  also  n.sed  to  simulate  a  Fonrii'r  filler.  Using  a  Koho¬ 
nen  net,  a  MFD,  a  coiinIcrpmpagHtion  and  a  hybrid  network,  data  measured  from 
infrared  and  laser  ratlar  iinagerj*  of  militar}’  vehicles  is  analyzed.  The  accuracy  and 
training  times  for  a  MFD  net  and  a  Hybrid  net  arc  compared  using  an  ambiguous 
dcTisioii  region  problem.  Each  classification  problem  is  examined  aud  compared  to 
classical,  nearest  neighbor  |>attcm  recognition  technii|ues.  Using  dynamic  analysis, 
neural  network  pruning  is  used  to  determine  opt  iiiiuin  node  coungiirations.  A  hybrid 
neural  network  is  developed  using  Kohonen  training  rules  for  the  first  Iiidilen  layer 
followed  by  one  or  two  hidden  layers  using  standard  l>ack  propagal  ion  rules  for  train¬ 
ing.  Advantage  of  the  hybrid  network  is  shown  for  class!  Rest  ion  |>n>bleins  in\x>lviiig 
anomalies  characteristic  of  measured  data.  The  Hybrid  network  retpiires  less  train¬ 
ing  and  fewer  interconnections  than  MFD  when  classiRcations  involves  anibignons 
decision  regions. 


